-- Logs begin at Mon 2022-10-24 08:03:29 UTC, end at Mon 2022-10-24 09:06:42 UTC. -- Oct 24 08:33:47.277224 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Oct 24 08:33:47.287679 dv-cinder-master-3-smb devstack@keystone.service[87859]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 24 08:33:47 2022] *** Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: nodename: dv-cinder-master-3-smb Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: machine: x86_64 Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: clock source: unix Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: pcre jit disabled Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: detected number of CPU cores: 4 Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: current working directory: / Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: detected binary path: /usr/bin/uwsgi-core Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: your processes number limit is 47680 Oct 24 08:33:47.290083 dv-cinder-master-3-smb devstack@keystone.service[87859]: your memory page size is 4096 bytes Oct 24 08:33:47.290849 dv-cinder-master-3-smb devstack@keystone.service[87859]: detected max file descriptor number: 2048 Oct 24 08:33:47.290849 dv-cinder-master-3-smb devstack@keystone.service[87859]: lock engine: pthread robust mutexes Oct 24 08:33:47.290849 dv-cinder-master-3-smb devstack@keystone.service[87859]: thunder lock: enabled Oct 24 08:33:47.290849 dv-cinder-master-3-smb devstack@keystone.service[87859]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 24 08:33:47.290849 dv-cinder-master-3-smb devstack@keystone.service[87859]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 24 08:33:47.328727 dv-cinder-master-3-smb devstack@keystone.service[87859]: Python main interpreter initialized at 0x5631ca7ee9c0 Oct 24 08:33:47.328727 dv-cinder-master-3-smb devstack@keystone.service[87859]: python threads support enabled Oct 24 08:33:47.328727 dv-cinder-master-3-smb devstack@keystone.service[87859]: your server socket listen backlog is limited to 100 connections Oct 24 08:33:47.328727 dv-cinder-master-3-smb devstack@keystone.service[87859]: your mercy for graceful operations on workers is 90 seconds Oct 24 08:33:47.329078 dv-cinder-master-3-smb devstack@keystone.service[87859]: mapped 403077 bytes (393 KB) for 2 cores Oct 24 08:33:47.329078 dv-cinder-master-3-smb devstack@keystone.service[87859]: *** Operational MODE: preforking *** Oct 24 08:33:47.329384 dv-cinder-master-3-smb devstack@keystone.service[87859]: *** uWSGI is running in multiple interpreter mode *** Oct 24 08:33:47.329384 dv-cinder-master-3-smb devstack@keystone.service[87859]: spawned uWSGI master process (pid: 87859) Oct 24 08:33:47.329539 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Oct 24 08:33:47.330237 dv-cinder-master-3-smb devstack@keystone.service[87859]: spawned uWSGI worker 1 (pid: 87872, cores: 1) Oct 24 08:33:47.332432 dv-cinder-master-3-smb devstack@keystone.service[87859]: spawned uWSGI worker 2 (pid: 87873, cores: 1) Oct 24 08:33:47.332432 dv-cinder-master-3-smb devstack@keystone.service[87859]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 24 08:33:51.517692 dv-cinder-master-3-smb devstack@keystone.service[87872]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:33:51.517692 dv-cinder-master-3-smb devstack@keystone.service[87872]: from cryptography.utils import int_from_bytes Oct 24 08:33:51.518536 dv-cinder-master-3-smb devstack@keystone.service[87873]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:33:51.518536 dv-cinder-master-3-smb devstack@keystone.service[87873]: from cryptography.utils import int_from_bytes Oct 24 08:33:51.518536 dv-cinder-master-3-smb devstack@keystone.service[87873]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:33:51.518536 dv-cinder-master-3-smb devstack@keystone.service[87873]: from cryptography.utils import int_from_bytes Oct 24 08:33:51.519102 dv-cinder-master-3-smb devstack@keystone.service[87872]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 08:33:51.519102 dv-cinder-master-3-smb devstack@keystone.service[87872]: from cryptography.utils import int_from_bytes Oct 24 08:33:51.567433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 08:33:51.567615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 08:33:51.567615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 08:33:51.567768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 08:33:51.567835 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] command line args: [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 08:33:51.567898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] command line args: [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 08:33:51.567956 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 08:33:51.568023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 08:33:51.568087 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ================================================================================ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 08:33:51.568150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ================================================================================ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 08:33:51.568328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] admin_token = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.568409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] admin_token = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.568569 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] config_dir = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.568666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] config_dir = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.568739 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.568805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.568935 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] config_source = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.569078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] config_source = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.569078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] debug = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.569213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] debug = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.569272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.570876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.570876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571106 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] log_config_append = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] log_config_append = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] log_dir = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] log_dir = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.571942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] log_file = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572005 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] log_file = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572318 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572499 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572723 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572888 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.572952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573009 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573622 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573681 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.573920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.574032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.574173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.574253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.574398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.574499 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.574930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] publish_errors = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] publish_errors = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575492 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575756 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.575916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.576023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.576175 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.576294 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.576382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] standard_threads = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.576460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] standard_threads = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.576610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.576748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.576853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.576945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] use_journal = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] use_journal = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] use_json = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] use_stderr = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] use_json = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] use_syslog = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] use_stderr = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] use_syslog = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.577954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.578035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 08:33:51.578111 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.578313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.578434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.578524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.578597 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.578827 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.578948 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.579025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.579304 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.579437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.579517 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.579588 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.579771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.579902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.579981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.580062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.580234 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.580357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.580435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.580515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.580717 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.580852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.580932 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.581003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.581179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.581298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.581384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.581468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.581627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.581713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.581869 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.581951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.582098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.582186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.582346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.582427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.582594 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.582679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.582855 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.583024 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.583104 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.583182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.583320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.583418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.583572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.583652 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.583816 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.583895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.584059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.584139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.584283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.584363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.584504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.584589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.584728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.584815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.584957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.585037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.585195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.585275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.585441 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.585526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.585658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.585738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.585881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.585961 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.586102 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.586179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.586344 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.586420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.586576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.586651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.586840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.586930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.587063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.587139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.587294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.587375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.587522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.587603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.587746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.587827 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.587971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.588045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.588195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.588271 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.588416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.588488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.588638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.588713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.588880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.588955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.589107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.589188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.589338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.589419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.589560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.589635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.589789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.589865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.590020 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.590103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.590260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.590336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.590486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.590566 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.590755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.590834 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.590971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.591047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.591203 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.591306 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.591432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.591565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.591669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.591770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.591870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.591968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.592112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.592233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.592433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.592556 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.592667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.592798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.592940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.593118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.593292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.593466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.593631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.593810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.593974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.594137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.594272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.594436 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.594602 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.594800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.594968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.595113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.595278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.595440 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.595578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.595751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.595916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.596084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.596221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.596386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.596548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.596723 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.596867 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.597006 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.597118 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.597251 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.597363 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.597484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.597624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.597738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.597853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.597969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.598086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.598208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.598344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.598473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.598591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.598706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.598866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.598988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.599100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.599209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.599318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.599427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.599536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.599683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.599798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.599908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.600018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.600134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.600245 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.600362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.600472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.600582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.600691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.600802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.600916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.601102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.601215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.601325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.601434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.601545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.601670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.601789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.601910 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.602021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.602131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.602247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.602420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.602533 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.602655 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.602800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.602916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.603032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.603144 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.603253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.603370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.603483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.603593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.603702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.603812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.603921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.604043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.604154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.604270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.604381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.604490 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.604629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.604753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.external = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.604872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.external = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.604993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.605104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.605215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.605326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.605438 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.605547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.605665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.password = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.605785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.password = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.605897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.token = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.606014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.token = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.606127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.606237 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.606355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.606466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.606576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.606686 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.606846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.606957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.607079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.607195 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.607306 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.607415 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.607531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.607642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.607760 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.607871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.607981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.608091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.608202 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.608312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.608431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.608542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.608652 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.608761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.608871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.608980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.609098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.609211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.609320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.609430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.609546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.609657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.609775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.609886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.609997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.610106 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.610222 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.610335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.610455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.610566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.610676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.610804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.610920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.611030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.611152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.611264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.611383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.611604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.611722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.611840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.611952 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.612061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.612173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.612283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.612394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.612515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.612627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.612737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.612847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.612957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.613074 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.613185 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.613296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.613407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.613517 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.613627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.613749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.613861 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.613970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.614079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.614189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.614299 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.614488 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.614602 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.614741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.614857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.614968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.615078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.615201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.615313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.615313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.615522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.615522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.615729 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.615847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.615959 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.616070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.616180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.616180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.616386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.616504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.616616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.616727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.616837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.616947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.617928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.618145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.618145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.618350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.618468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.618580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.618691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.618839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.618950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.619060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.619181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.619294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.619404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.619514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.619625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.619736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.619854 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.619965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.620075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.620185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.620295 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.620412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.620532 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.620643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.620753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.620866 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.620977 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.621087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.621210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.621322 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.621432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.621541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.621651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.621762 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.621880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.621991 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.622100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.622210 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.622320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.622437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.622548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.622658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.622788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.622901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.623012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.623123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.623123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.623331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.623331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.623538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.623708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.623833 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.623946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.624056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.624166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.624275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.624385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.624503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.624615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.624725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.624834 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.624945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.625054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.625172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.625283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.625392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.625502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.625620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.625731 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.625849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.625961 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.626078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.626189 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.626299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.626409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.626526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.626644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.626768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.626880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.626990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.627101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.627223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.627335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.627445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.627555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.627664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.627774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.627892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.628003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.628115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.628224 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.628334 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.628443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.628561 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.628672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.628782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.628891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.629001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.629112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.629230 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.629340 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.629451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.629566 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.629677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.629787 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.629904 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.630015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.630127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.630238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.630347 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.630456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.630606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.630798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.630917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.631027 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.631137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.631248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.631370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.631483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.631593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.631703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.631813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.631923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.632041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.632152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.632263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.632373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.632482 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.632592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.632709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.632820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.632931 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.633040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.633150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.633260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.633379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.633497 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.633607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.633719 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.633828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.633941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.634059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.634170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.634280 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.634391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.634500 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.634610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.634745 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.634859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.634969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.635079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.635189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.635298 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.635421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.635533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.635642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.635751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.635861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.635970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.636156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.636269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] role.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.636379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.636490 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] role.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.636599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] role.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.636716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.636827 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] role.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.636936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.637045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.637154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.637264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.637381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.637493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.637603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.637712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.637821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.637930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.638047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.638158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.638268 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.638377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.638486 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.638597 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.638732 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.638845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.638955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.639064 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.639173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.639282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.639403 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.639515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.639626 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.639736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.639847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.639955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640294 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.640969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.641175 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.641293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.641405 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.641515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.641624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.641735 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.641845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.641963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.642073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.642184 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.642294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.642403 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.642512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.642629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.642758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.642871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.642980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.643090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.643199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.643328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.643440 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.643550 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.643659 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.643779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.643890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.644008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] token.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.644119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.644229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.644339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.644449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.644558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] token.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.644675 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.644788 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.644899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.645008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.645118 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.645227 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.645345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.645456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.645566 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.645676 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.645785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.645897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.646015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.646126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.646235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.646345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.646454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.646563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.646684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.646880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.646993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.647102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.647212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.647321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.647443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.647564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.647683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.647794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.647904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.648111 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.648240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.648361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.648473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.648592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.648703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.648813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.648923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.649034 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.649144 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.649262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.649374 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.649483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.649593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.649703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.649813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.649931 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.650043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.650154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.650264 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.650375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.650488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.650606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.650736 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.650853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.650965 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.651075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.651196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.651320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.651432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.651542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.651652 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.651761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.651871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.651988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.652100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.652211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.652321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.652431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.652540 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 08:33:51.652659 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.652659 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 08:33:51.652659 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 08:33:51.679769 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.679995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.680158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.680376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.680484 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.680606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.688700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.688885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.716227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.716441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.716563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.716685 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.768749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.769140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.817047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.817201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.817342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.817422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.975376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.975599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.975780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.976018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.976154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.976259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.976325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.976387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.976447 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.976665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.976880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.977096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.999123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.999251 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.999329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.999457 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.999558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.999683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.999764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:51.999941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.000066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.000157 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.000222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.000417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.000544 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.000682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.011370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.011682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.024338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.024604 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.024793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.025025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.025261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.025387 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.025465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.025528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.025729 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.025865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.026000 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.026159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.026397 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.026613 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 08:33:52.042786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.043009 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 08:33:52.043131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.043265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 08:33:52.043343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.043537 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 08:33:52.043678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.043762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 08:33:52.043835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.044049 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.044172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.044319 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.044400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.044549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.044672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.044805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.054229 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.054507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.054629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.054784 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.054866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.055082 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.055339 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.055339 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.062169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.062477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.065099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.065380 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.065501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.065629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.065725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.065927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.066052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.066191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.066277 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.066453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.066576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.066695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.066828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.066953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.067030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.067254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.067380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.067552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.067635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.067754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.067843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.068065 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.090364 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.090847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.091850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.092136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.092331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.092646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.096882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.097164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.097366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.097489 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.097624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.097919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.103878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.104162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.104386 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.104674 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.109886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.110182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.110472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.110595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.110757 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.110840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.111036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.111160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.111294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.111373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.111579 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.111716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.111851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.111931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.112171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.112454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.127587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.127859 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.128290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.128572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.131729 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.132461 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.134980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.135279 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.135731 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.136029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.139532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.140298 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.142676 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.143493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.145900 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.146195 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.146483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.146704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.146851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.146983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.147066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.147302 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.147425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.147551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.147876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.148183 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.163450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.163733 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.164004 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.164261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.164399 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.164654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.164936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.165206 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.171222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.172146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.172750 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.173695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.174296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.174657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.175277 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.175646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.177340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.177633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.177901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.178195 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.178316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.178578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.178740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.178876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.179160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.179559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.189802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.190076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.190377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.190668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.190812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.191085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.191211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.191338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.191422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.191644 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.191767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.192045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.192341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.192621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.207065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.208127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.209968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.211076 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.212902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.214016 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.214332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.215513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.215837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.216128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.216522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.216998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.217317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.217735 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.222984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.224265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.224468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.225819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.227362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.227753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.228040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.228321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.228668 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.229065 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.229352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.229636 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.236171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.236451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.236727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.237550 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.237840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.238124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.243663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.243947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.244226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.244503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.244771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.245044 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.245166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 08:33:52.245312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.245405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.245591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.245716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.245851 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.245930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.246147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.246285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.246430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.246512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.246643 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.246752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.246978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.247101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.247207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.247353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.247521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.247793 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.248040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.248317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.248644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 08:33:52.281177 dv-cinder-master-3-smb devstack@keystone.service[87873]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5631ca7ee9c0 pid: 87873 (default app) Oct 24 08:33:52.282699 dv-cinder-master-3-smb devstack@keystone.service[87872]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5631ca7ee9c0 pid: 87872 (default app) Oct 24 08:33:52.284606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263ca40-0253-4587-b8bc-d928ed476064 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:33:52.284893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263ca40-0253-4587-b8bc-d928ed476064 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:33:52.285166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263ca40-0253-4587-b8bc-d928ed476064 None None] PATH_INFO: `/v3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:33:52.289733 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Mon Oct 24 08:33:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:33:58.973507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddba76-0b44-439f-bf84-6154607e931c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:33:58.974364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddba76-0b44-439f-bf84-6154607e931c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:33:58.974364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddba76-0b44-439f-bf84-6154607e931c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:33:58.981452 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:33:58 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:33:58.987424 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:33:58.988013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:33:58.988013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:33:59.059242 dv-cinder-master-3-smb devstack@keystone.service[87873]: INFO dbcounter [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] Registered counter for database keystone Oct 24 08:33:59.080110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG oslo_db.sqlalchemy.engines [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87873) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:33:59.092136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writer thread running {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:33:59.270812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG passlib.handlers.bcrypt [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87873) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 24 08:33:59.271142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG passlib.handlers.bcrypt [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87873) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 24 08:33:59.330692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:33:59.332357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:33:59.463548 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:33:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 08:33:59.997289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddba76-0b44-439f-bf84-6154607e931c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:00.068657 dv-cinder-master-3-smb devstack@keystone.service[87872]: INFO dbcounter [None req-35ddba76-0b44-439f-bf84-6154607e931c None None] Registered counter for database keystone Oct 24 08:34:00.093353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG oslo_db.sqlalchemy.engines [None req-35ddba76-0b44-439f-bf84-6154607e931c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87872) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 08:34:00.105467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writer thread running {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 08:34:00.223739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddba76-0b44-439f-bf84-6154607e931c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:00.229220 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38ddd9-3d7b-4704-b159-a3d30d97be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:00.229843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38ddd9-3d7b-4704-b159-a3d30d97be29 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:00.230119 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38ddd9-3d7b-4704-b159-a3d30d97be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:00.230389 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38ddd9-3d7b-4704-b159-a3d30d97be29 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:00.239710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38ddd9-3d7b-4704-b159-a3d30d97be29 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:00.290613 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6f38ddd9-3d7b-4704-b159-a3d30d97be29 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 24 08:34:00.290981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38ddd9-3d7b-4704-b159-a3d30d97be29 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:00.300962 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Oct 24 08:33:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:34:00.307177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:00.325969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f8f2d-0f10-477b-884b-be6670d4b4e5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:00.331652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3371a11-2d52-47ed-89ff-a5119e1d5047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:00.332776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3371a11-2d52-47ed-89ff-a5119e1d5047 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:00.333057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3371a11-2d52-47ed-89ff-a5119e1d5047 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:00.333327 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3371a11-2d52-47ed-89ff-a5119e1d5047 admin admin] PATH_INFO: `/v3/projects/d2f48107e8bb4875908f0d6dc2693154` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:00.335072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3371a11-2d52-47ed-89ff-a5119e1d5047 admin admin] RBAC: Authorizing `identity:get_project(project_id=d2f48107e8bb4875908f0d6dc2693154)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:00.389566 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c3371a11-2d52-47ed-89ff-a5119e1d5047 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 24 08:34:00.389908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3371a11-2d52-47ed-89ff-a5119e1d5047 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:00.394454 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Oct 24 08:34:00 2022] GET /identity/v3/projects/d2f48107e8bb4875908f0d6dc2693154 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:02.507560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd51c1-e800-4b5a-a155-dd307c14a372 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:02.508474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd51c1-e800-4b5a-a155-dd307c14a372 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:02.508474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd51c1-e800-4b5a-a155-dd307c14a372 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:02.514990 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:02.520731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c283b6-0863-401f-80df-463c779a9e5a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:02.521170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c283b6-0863-401f-80df-463c779a9e5a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:02.521263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c283b6-0863-401f-80df-463c779a9e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:02.543290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-12c283b6-0863-401f-80df-463c779a9e5a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:02.544717 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-12c283b6-0863-401f-80df-463c779a9e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:02.553486 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 08:34:02.818562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd51c1-e800-4b5a-a155-dd307c14a372 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:02.832904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd51c1-e800-4b5a-a155-dd307c14a372 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:02.838166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f890d-6f3f-4a5e-bb1e-c3fdeb1b4a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:02.838790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f890d-6f3f-4a5e-bb1e-c3fdeb1b4a33 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:02.839071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f890d-6f3f-4a5e-bb1e-c3fdeb1b4a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:02.839337 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f890d-6f3f-4a5e-bb1e-c3fdeb1b4a33 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:02.847920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f890d-6f3f-4a5e-bb1e-c3fdeb1b4a33 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:02.849846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f890d-6f3f-4a5e-bb1e-c3fdeb1b4a33 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:02.859668 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Oct 24 08:34:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:34:02.865968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c283b6-0863-401f-80df-463c779a9e5a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:02.878458 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c283b6-0863-401f-80df-463c779a9e5a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:02.884187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7599b6-77e0-4436-b4e8-20ee2680741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:02.884814 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7599b6-77e0-4436-b4e8-20ee2680741a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:02.885089 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7599b6-77e0-4436-b4e8-20ee2680741a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:02.885353 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7599b6-77e0-4436-b4e8-20ee2680741a admin admin] PATH_INFO: `/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:02.886829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7599b6-77e0-4436-b4e8-20ee2680741a admin admin] RBAC: Authorizing `identity:get_user(user_id=2e8c1f674ea94c1eb2d8a47d1aad00af)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:02.888658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7599b6-77e0-4436-b4e8-20ee2680741a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:02.890222 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Mon Oct 24 08:34:02 2022] GET /identity/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:05.044639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9599abc-2a28-45f3-b1e6-78c7b522b43f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:05.045190 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9599abc-2a28-45f3-b1e6-78c7b522b43f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:05.045190 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9599abc-2a28-45f3-b1e6-78c7b522b43f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:05.046461 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:05.051741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b82202-75ee-4717-ad80-cc6b6f374ed8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:05.052179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b82202-75ee-4717-ad80-cc6b6f374ed8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:05.052271 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b82202-75ee-4717-ad80-cc6b6f374ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:05.070557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-91b82202-75ee-4717-ad80-cc6b6f374ed8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:05.071759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-91b82202-75ee-4717-ad80-cc6b6f374ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:05.111775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffce16-33ae-4b53-ba17-54d942f5372c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:05.112031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffce16-33ae-4b53-ba17-54d942f5372c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:05.112307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffce16-33ae-4b53-ba17-54d942f5372c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:05.113637 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:05.118910 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b65d2-f0ab-4c72-b0f1-8a0bfa3ae6d1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:05.119196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b65d2-f0ab-4c72-b0f1-8a0bfa3ae6d1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:05.119473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b65d2-f0ab-4c72-b0f1-8a0bfa3ae6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:05.156718 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:05.161353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG passlib.handlers.bcrypt [None req-329b65d2-f0ab-4c72-b0f1-8a0bfa3ae6d1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87872) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 24 08:34:05.161787 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG passlib.handlers.bcrypt [None req-329b65d2-f0ab-4c72-b0f1-8a0bfa3ae6d1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87872) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 24 08:34:05.210471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-329b65d2-f0ab-4c72-b0f1-8a0bfa3ae6d1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:05.212064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-329b65d2-f0ab-4c72-b0f1-8a0bfa3ae6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:05.318645 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:05.427642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b82202-75ee-4717-ad80-cc6b6f374ed8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:05.452234 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:05.535162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b82202-75ee-4717-ad80-cc6b6f374ed8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:05.569529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca96e0-d949-44e5-b219-bb7681294b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:05.570109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca96e0-d949-44e5-b219-bb7681294b71 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:05.570391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca96e0-d949-44e5-b219-bb7681294b71 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:05.570669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca96e0-d949-44e5-b219-bb7681294b71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:05.580512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b65d2-f0ab-4c72-b0f1-8a0bfa3ae6d1 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:05.582622 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca96e0-d949-44e5-b219-bb7681294b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2aca96e0-d949-44e5-b219-bb7681294b71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:05.592513 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:05.593495 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:05.594344 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:34:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:05.600631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca96e0-d949-44e5-b219-bb7681294b71 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:05.680358 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:05.683695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b65d2-f0ab-4c72-b0f1-8a0bfa3ae6d1 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:05.686066 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:05.698410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca96e0-d949-44e5-b219-bb7681294b71 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:05.719721 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:05.720592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:05.720875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:05.721149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:05.722120 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:05.723623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:05.730942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b9325-46c4-480d-b726-55fb452e6b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:05.731532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b9325-46c4-480d-b726-55fb452e6b16 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:05.731810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b9325-46c4-480d-b726-55fb452e6b16 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:05.732083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b9325-46c4-480d-b726-55fb452e6b16 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:05.732778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b9325-46c4-480d-b726-55fb452e6b16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:05.733169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b9325-46c4-480d-b726-55fb452e6b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:05.734463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b9325-46c4-480d-b726-55fb452e6b16 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:05.735226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.sql.core [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 08:34:05.735226 dv-cinder-master-3-smb devstack@keystone.service[87872]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 08:34:05.735226 dv-cinder-master-3-smb devstack@keystone.service[87872]: [parameters: {'id': '9e19aed7a3254e8ea844e4197b5d039b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 08:34:05.735226 dv-cinder-master-3-smb devstack@keystone.service[87872]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87872) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 08:34:05.746063 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:05.752024 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b9325-46c4-480d-b726-55fb452e6b16 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:34:05.754140 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application result.read() Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 08:34:05.755005 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application yield resource Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise self.value Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application session.commit() Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 08:34:05.755918 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.session.flush() Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application compat.raise_( Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 08:34:05.756776 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 08:34:05.757652 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application result.read() Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:34:05.758407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [parameters: {'id': '9e19aed7a3254e8ea844e4197b5d039b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 08:34:05.760066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:05.761359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 8/16] 10.222.0.26 () {60 vars in 1137 bytes} [Mon Oct 24 08:34:05 2022] POST /identity/v3/domains => generated 190 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:34:05.762781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:05.852429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b9325-46c4-480d-b726-55fb452e6b16 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:05.862345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e1f5b-d8e3-4b58-8666-4cc131459dba None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:05.884654 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da25723-c000-4657-b8ed-07ed6d88cd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:05.885290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da25723-c000-4657-b8ed-07ed6d88cd72 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:05.885569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da25723-c000-4657-b8ed-07ed6d88cd72 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:05.885838 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da25723-c000-4657-b8ed-07ed6d88cd72 None admin] PATH_INFO: `/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:05.886998 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da25723-c000-4657-b8ed-07ed6d88cd72 None admin] RBAC: Authorizing `identity:get_user(user_id=2e8c1f674ea94c1eb2d8a47d1aad00af)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:05.888467 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da25723-c000-4657-b8ed-07ed6d88cd72 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:05.890130 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:05 2022] GET /identity/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:05.895281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969d741-f4f7-41f6-9c9b-b6ddf3ee2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:05.895878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da25723-c000-4657-b8ed-07ed6d88cd72 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:05.896136 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969d741-f4f7-41f6-9c9b-b6ddf3ee2b0d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:05.896418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969d741-f4f7-41f6-9c9b-b6ddf3ee2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:05.896697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969d741-f4f7-41f6-9c9b-b6ddf3ee2b0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:05.909706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:05.914870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969d741-f4f7-41f6-9c9b-b6ddf3ee2b0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:05.916358 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969d741-f4f7-41f6-9c9b-b6ddf3ee2b0d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1969d741-f4f7-41f6-9c9b-b6ddf3ee2b0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:34:05.932128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:05.933212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:34:05.934377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:34:05.934377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:34:05.934377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:05.934377 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:34:05.939174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969d741-f4f7-41f6-9c9b-b6ddf3ee2b0d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:05.962578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:05.995957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da25723-c000-4657-b8ed-07ed6d88cd72 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:06.030382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07b1ab-c757-4e52-8601-f1158985bd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:06.031045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07b1ab-c757-4e52-8601-f1158985bd27 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:06.031328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07b1ab-c757-4e52-8601-f1158985bd27 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:06.031592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07b1ab-c757-4e52-8601-f1158985bd27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:06.032704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07b1ab-c757-4e52-8601-f1158985bd27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:06.034201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07b1ab-c757-4e52-8601-f1158985bd27 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:06.035778 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:06.039867 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969d741-f4f7-41f6-9c9b-b6ddf3ee2b0d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:06.041894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07b1ab-c757-4e52-8601-f1158985bd27 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:06.076719 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42475e8-4a83-41f5-8156-764b52848d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:06.077326 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42475e8-4a83-41f5-8156-764b52848d6b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:06.077611 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42475e8-4a83-41f5-8156-764b52848d6b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:06.077883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42475e8-4a83-41f5-8156-764b52848d6b None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:06.078528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42475e8-4a83-41f5-8156-764b52848d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:06.078881 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42475e8-4a83-41f5-8156-764b52848d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:06.080276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42475e8-4a83-41f5-8156-764b52848d6b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:06.097534 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 10/20] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:06.121619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:06.144095 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07b1ab-c757-4e52-8601-f1158985bd27 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:06.183777 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97828d-9f14-49dc-af60-f3deaab2baa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:06.184481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97828d-9f14-49dc-af60-f3deaab2baa4 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:06.184763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97828d-9f14-49dc-af60-f3deaab2baa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:06.185044 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97828d-9f14-49dc-af60-f3deaab2baa4 None admin] PATH_INFO: `/v3/domains/default/users/2e8c1f674ea94c1eb2d8a47d1aad00af/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:06.187350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97828d-9f14-49dc-af60-f3deaab2baa4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2e8c1f674ea94c1eb2d8a47d1aad00af, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:06.188828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97828d-9f14-49dc-af60-f3deaab2baa4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:06.304753 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 11/21] 10.222.0.26 () {58 vars in 1361 bytes} [Mon Oct 24 08:34:06 2022] PUT /identity/v3/domains/default/users/2e8c1f674ea94c1eb2d8a47d1aad00af/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:08.591273 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b476559-1b29-4d25-ab61-8e461d40b17c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:08.592424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b476559-1b29-4d25-ab61-8e461d40b17c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:08.592900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b476559-1b29-4d25-ab61-8e461d40b17c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:08.594434 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 11/22] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:08.600717 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5050c-5e0c-41dc-8e96-52e33223e159 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:08.601650 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5050c-5e0c-41dc-8e96-52e33223e159 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:08.602122 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5050c-5e0c-41dc-8e96-52e33223e159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:08.631842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-72f5050c-5e0c-41dc-8e96-52e33223e159 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:08.633709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-72f5050c-5e0c-41dc-8e96-52e33223e159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:08.711358 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 12/23] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:08.950706 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4957f-2ec7-4967-b04e-c43181916204 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:08.951456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4957f-2ec7-4967-b04e-c43181916204 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:08.951947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4957f-2ec7-4967-b04e-c43181916204 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:08.953756 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 12/24] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:08.958849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143791-bbcc-43cf-a7fa-4bb62b480307 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:08.959363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143791-bbcc-43cf-a7fa-4bb62b480307 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:08.959845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143791-bbcc-43cf-a7fa-4bb62b480307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:08.987863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-01143791-bbcc-43cf-a7fa-4bb62b480307 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:08.989394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-01143791-bbcc-43cf-a7fa-4bb62b480307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:09.011751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f91550-025e-4e47-9cf4-6b715d8f34a1 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:09.012411 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f91550-025e-4e47-9cf4-6b715d8f34a1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:09.012883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f91550-025e-4e47-9cf4-6b715d8f34a1 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:09.014783 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 13/25] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:09.024294 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4bff9-3708-4fce-87b2-c73685828f4f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:09.024853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4bff9-3708-4fce-87b2-c73685828f4f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:09.025333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4bff9-3708-4fce-87b2-c73685828f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:09.054995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-50e4bff9-3708-4fce-87b2-c73685828f4f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:09.056593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-50e4bff9-3708-4fce-87b2-c73685828f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:09.091026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:09.099540 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 13/26] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:09.119411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa5927-1c2d-421f-b124-76ae9366ad60 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:09.119715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa5927-1c2d-421f-b124-76ae9366ad60 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:09.119963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa5927-1c2d-421f-b124-76ae9366ad60 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:09.124336 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 14/27] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:34:09.129474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127e313-dfb5-4246-a127-8d1e8a9a02f7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:09.129760 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127e313-dfb5-4246-a127-8d1e8a9a02f7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:09.130040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127e313-dfb5-4246-a127-8d1e8a9a02f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:09.164361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4127e313-dfb5-4246-a127-8d1e8a9a02f7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:09.165578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4127e313-dfb5-4246-a127-8d1e8a9a02f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:09.196596 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 14/28] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:09.197952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4bff9-3708-4fce-87b2-c73685828f4f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:09.234139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:09.277614 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 15/29] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:09.405545 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4bff9-3708-4fce-87b2-c73685828f4f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:09.439248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666888a-1d71-4097-87da-1ebbf07033f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:09.439887 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666888a-1d71-4097-87da-1ebbf07033f1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:09.440163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666888a-1d71-4097-87da-1ebbf07033f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:09.440440 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666888a-1d71-4097-87da-1ebbf07033f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:09.450413 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666888a-1d71-4097-87da-1ebbf07033f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:09.451963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666888a-1d71-4097-87da-1ebbf07033f1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9666888a-1d71-4097-87da-1ebbf07033f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:34:09.466380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:09.467394 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:34:09.468217 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:34:09.468217 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:34:09.468217 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:09.468217 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 15/30] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:09.475242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127e313-dfb5-4246-a127-8d1e8a9a02f7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:09.525309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666888a-1d71-4097-87da-1ebbf07033f1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:09.612653 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127e313-dfb5-4246-a127-8d1e8a9a02f7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:09.667096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:09.669178 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:09.673609 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e5a89-67a4-468e-8eda-00c39d6503fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:09.674239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e5a89-67a4-468e-8eda-00c39d6503fc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:09.674539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e5a89-67a4-468e-8eda-00c39d6503fc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:09.675085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e5a89-67a4-468e-8eda-00c39d6503fc None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:09.675938 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e5a89-67a4-468e-8eda-00c39d6503fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:09.676475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e5a89-67a4-468e-8eda-00c39d6503fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:09.678151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e5a89-67a4-468e-8eda-00c39d6503fc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:09.697638 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 16/31] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:09.698328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e5a89-67a4-468e-8eda-00c39d6503fc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:09.702943 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666888a-1d71-4097-87da-1ebbf07033f1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:09.737890 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4200a9-88e9-439f-b0b2-7d330b955bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:09.738509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4200a9-88e9-439f-b0b2-7d330b955bc7 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:09.738844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4200a9-88e9-439f-b0b2-7d330b955bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:09.739122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4200a9-88e9-439f-b0b2-7d330b955bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:09.739811 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4200a9-88e9-439f-b0b2-7d330b955bc7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:09.741638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4200a9-88e9-439f-b0b2-7d330b955bc7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:09.808083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e5a89-67a4-468e-8eda-00c39d6503fc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:09.847233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b903d3-712a-4d3f-89a8-fcdc6cb62718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:09.847856 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b903d3-712a-4d3f-89a8-fcdc6cb62718 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:09.848176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b903d3-712a-4d3f-89a8-fcdc6cb62718 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:09.848506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b903d3-712a-4d3f-89a8-fcdc6cb62718 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:09.849186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b903d3-712a-4d3f-89a8-fcdc6cb62718 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:09.850586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b903d3-712a-4d3f-89a8-fcdc6cb62718 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:09.868953 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 17/32] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Oct 24 08:34:09 2022] POST /identity/v3/roles => generated 219 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:09.869903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b903d3-712a-4d3f-89a8-fcdc6cb62718 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:09.880838 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 16/33] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Oct 24 08:34:09 2022] POST /identity/v3/roles => generated 225 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:34:09.881754 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4200a9-88e9-439f-b0b2-7d330b955bc7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:09.941775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:10.004614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4200a9-88e9-439f-b0b2-7d330b955bc7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:10.019699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b903d3-712a-4d3f-89a8-fcdc6cb62718 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:10.032268 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:10.048425 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb2d74-4b6c-45bf-a56c-8c13a5609fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:10.049271 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb2d74-4b6c-45bf-a56c-8c13a5609fbc None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.049684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb2d74-4b6c-45bf-a56c-8c13a5609fbc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.050060 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb2d74-4b6c-45bf-a56c-8c13a5609fbc None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.050915 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb2d74-4b6c-45bf-a56c-8c13a5609fbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:10.052682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb2d74-4b6c-45bf-a56c-8c13a5609fbc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:10.068211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce68d5-922e-43d1-9db7-75dc0f94bcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:10.068823 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce68d5-922e-43d1-9db7-75dc0f94bcf0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.069110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce68d5-922e-43d1-9db7-75dc0f94bcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.069398 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce68d5-922e-43d1-9db7-75dc0f94bcf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.083087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce68d5-922e-43d1-9db7-75dc0f94bcf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-dbce68d5-922e-43d1-9db7-75dc0f94bcf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:10.089110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:10.091396 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:10.093174 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 18/34] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:34:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:10.100166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce68d5-922e-43d1-9db7-75dc0f94bcf0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:10.109258 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 17/35] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:34:09 2022] POST /identity/v3/projects => generated 301 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:10.242290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce68d5-922e-43d1-9db7-75dc0f94bcf0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:10.254324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:10.281787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e21277-663a-48bb-ab9d-dc4133c2206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:10.282118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e21277-663a-48bb-ab9d-dc4133c2206a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.282400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e21277-663a-48bb-ab9d-dc4133c2206a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.282678 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e21277-663a-48bb-ab9d-dc4133c2206a None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.283453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e21277-663a-48bb-ab9d-dc4133c2206a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:10.283807 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e21277-663a-48bb-ab9d-dc4133c2206a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:10.285248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e21277-663a-48bb-ab9d-dc4133c2206a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:10.295881 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 19/36] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:10.301798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb2d74-4b6c-45bf-a56c-8c13a5609fbc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:10.419382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb2d74-4b6c-45bf-a56c-8c13a5609fbc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:10.460883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c4c94-73a2-499b-bcfb-3d3c925b146d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:10.461929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c4c94-73a2-499b-bcfb-3d3c925b146d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.462406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c4c94-73a2-499b-bcfb-3d3c925b146d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.462960 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c4c94-73a2-499b-bcfb-3d3c925b146d None admin] PATH_INFO: `/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.464640 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c4c94-73a2-499b-bcfb-3d3c925b146d None admin] RBAC: Authorizing `identity:get_user(user_id=2e8c1f674ea94c1eb2d8a47d1aad00af)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:10.466098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c4c94-73a2-499b-bcfb-3d3c925b146d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:10.468066 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 18/37] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:10 2022] GET /identity/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:10.475981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e21277-663a-48bb-ab9d-dc4133c2206a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:10.528933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9d3c-73a3-4cf7-befa-d013a10a3b67 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.529234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9d3c-73a3-4cf7-befa-d013a10a3b67 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.529534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9d3c-73a3-4cf7-befa-d013a10a3b67 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.531084 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 19/38] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:34:10.535940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f366f-e9fb-41e1-8403-6c0fcc4457a4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.536229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f366f-e9fb-41e1-8403-6c0fcc4457a4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.536599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f366f-e9fb-41e1-8403-6c0fcc4457a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.555272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-213f366f-e9fb-41e1-8403-6c0fcc4457a4 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:10.556821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-213f366f-e9fb-41e1-8403-6c0fcc4457a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:10.596661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e21277-663a-48bb-ab9d-dc4133c2206a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:10.600315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:10.637363 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 20/39] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:10.693962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c703d75-a59c-40fc-8a7e-0b9446a9fd7d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.694562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c703d75-a59c-40fc-8a7e-0b9446a9fd7d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.694562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c703d75-a59c-40fc-8a7e-0b9446a9fd7d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.695926 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 21/40] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:10.701963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc999b-d13e-4f64-99f5-babb6c6ff7d3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.702245 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc999b-d13e-4f64-99f5-babb6c6ff7d3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.702523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc999b-d13e-4f64-99f5-babb6c6ff7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.722046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bdfc999b-d13e-4f64-99f5-babb6c6ff7d3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:10.723260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bdfc999b-d13e-4f64-99f5-babb6c6ff7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:10.749190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289332f-27ab-417b-8f38-4eac159feb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:10.749841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289332f-27ab-417b-8f38-4eac159feb94 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.750133 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289332f-27ab-417b-8f38-4eac159feb94 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.750410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289332f-27ab-417b-8f38-4eac159feb94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.751582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289332f-27ab-417b-8f38-4eac159feb94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:10.753053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289332f-27ab-417b-8f38-4eac159feb94 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:10.754762 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 20/41] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:10.761233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289332f-27ab-417b-8f38-4eac159feb94 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:10.793900 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 22/42] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:10.803307 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:10.866732 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289332f-27ab-417b-8f38-4eac159feb94 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:10.900515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30468df1-531d-4549-b8cc-3d77e015a282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:10.901162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30468df1-531d-4549-b8cc-3d77e015a282 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:10.901448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30468df1-531d-4549-b8cc-3d77e015a282 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:10.901727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30468df1-531d-4549-b8cc-3d77e015a282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:10.902453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30468df1-531d-4549-b8cc-3d77e015a282 None admin] RBAC: Adding query filter params (role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.domain.id=default, user.id=2e8c1f674ea94c1eb2d8a47d1aad00af) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:10.902772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30468df1-531d-4549-b8cc-3d77e015a282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:10.903180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc999b-d13e-4f64-99f5-babb6c6ff7d3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:10.904235 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30468df1-531d-4549-b8cc-3d77e015a282 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:10.911842 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 21/43] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Oct 24 08:34:10 2022] GET /identity/v3/role_assignments?user.id=2e8c1f674ea94c1eb2d8a47d1aad00af&scope.domain.id=default&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:11.016472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc999b-d13e-4f64-99f5-babb6c6ff7d3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:11.018252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:11.058404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a561-b094-4267-9b01-21310b57b1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:11.059072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a561-b094-4267-9b01-21310b57b1b0 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:11.059359 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a561-b094-4267-9b01-21310b57b1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:11.059635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a561-b094-4267-9b01-21310b57b1b0 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:11.060338 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a561-b094-4267-9b01-21310b57b1b0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:11.061850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a561-b094-4267-9b01-21310b57b1b0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:11.062073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30468df1-531d-4549-b8cc-3d77e015a282 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:11.079105 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 23/44] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Oct 24 08:34:10 2022] POST /identity/v3/roles => generated 223 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:34:11.183212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30468df1-531d-4549-b8cc-3d77e015a282 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:11.186633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:11.221309 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734c8d6-3af8-4413-9bca-92d19a74edce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:11.221967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734c8d6-3af8-4413-9bca-92d19a74edce None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:11.222248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734c8d6-3af8-4413-9bca-92d19a74edce None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:11.222518 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734c8d6-3af8-4413-9bca-92d19a74edce None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:11.223750 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734c8d6-3af8-4413-9bca-92d19a74edce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:11.225240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734c8d6-3af8-4413-9bca-92d19a74edce None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:11.227107 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 22/45] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:11.232912 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a561-b094-4267-9b01-21310b57b1b0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:11.338654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a561-b094-4267-9b01-21310b57b1b0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:11.373205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e376ce-a3dd-43dd-a06c-d978a7b58fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:11.373802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e376ce-a3dd-43dd-a06c-d978a7b58fbe None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:11.374077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e376ce-a3dd-43dd-a06c-d978a7b58fbe None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:11.374353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e376ce-a3dd-43dd-a06c-d978a7b58fbe None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:11.375022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e376ce-a3dd-43dd-a06c-d978a7b58fbe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:11.376743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e376ce-a3dd-43dd-a06c-d978a7b58fbe None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:11.412007 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:34:11 2022] POST /identity/v3/projects => generated 312 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:13.511856 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbb462-9dc7-4b68-8c55-a9532da6db2b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:13.512684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbb462-9dc7-4b68-8c55-a9532da6db2b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:13.512684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbb462-9dc7-4b68-8c55-a9532da6db2b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:13.514580 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 23/47] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:13.520145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04806ba2-7f86-459f-b570-fedfb95f6201 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:13.520559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04806ba2-7f86-459f-b570-fedfb95f6201 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:13.520670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04806ba2-7f86-459f-b570-fedfb95f6201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:13.539705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-04806ba2-7f86-459f-b570-fedfb95f6201 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:13.540819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-04806ba2-7f86-459f-b570-fedfb95f6201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:13.549904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:13.609941 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 25/48] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:13.871806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbb462-9dc7-4b68-8c55-a9532da6db2b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:13.991017 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbb462-9dc7-4b68-8c55-a9532da6db2b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:14.025777 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b20433-f5fe-4f47-a506-5eeeb27c33fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:14.026401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20433-f5fe-4f47-a506-5eeeb27c33fc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:14.026682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20433-f5fe-4f47-a506-5eeeb27c33fc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:14.026989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20433-f5fe-4f47-a506-5eeeb27c33fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:14.028217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b20433-f5fe-4f47-a506-5eeeb27c33fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:14.029755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b20433-f5fe-4f47-a506-5eeeb27c33fc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:14.031355 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 24/49] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:14.040754 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04806ba2-7f86-459f-b570-fedfb95f6201 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:14.147669 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04806ba2-7f86-459f-b570-fedfb95f6201 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:14.195006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e31eb5-060f-4484-b531-68167400b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:14.195848 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e31eb5-060f-4484-b531-68167400b778 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:14.196231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e31eb5-060f-4484-b531-68167400b778 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:14.196565 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e31eb5-060f-4484-b531-68167400b778 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:14.197510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e31eb5-060f-4484-b531-68167400b778 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:14.199764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e31eb5-060f-4484-b531-68167400b778 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:14.216216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:14.232300 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 26/50] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Oct 24 08:34:14 2022] POST /identity/v3/projects => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:16.372013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bd4d2-51c3-4606-9237-6c775e90c23e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:16.372897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bd4d2-51c3-4606-9237-6c775e90c23e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:16.373041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bd4d2-51c3-4606-9237-6c775e90c23e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:16.374438 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 25/51] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:16.380113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d94a3-f954-4469-aa8c-cc4f18cb16e3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:16.380540 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d94a3-f954-4469-aa8c-cc4f18cb16e3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:16.380622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d94a3-f954-4469-aa8c-cc4f18cb16e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:16.399607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-563d94a3-f954-4469-aa8c-cc4f18cb16e3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:16.400774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-563d94a3-f954-4469-aa8c-cc4f18cb16e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:16.468159 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 27/52] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:16.734902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bd4d2-51c3-4606-9237-6c775e90c23e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:16.791835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:16.853102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bd4d2-51c3-4606-9237-6c775e90c23e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:16.887816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6acd6-189f-4c8a-9735-6851b54ad8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:16.888460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6acd6-189f-4c8a-9735-6851b54ad8f9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:16.888748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6acd6-189f-4c8a-9735-6851b54ad8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:16.889021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6acd6-189f-4c8a-9735-6851b54ad8f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:16.890187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6acd6-189f-4c8a-9735-6851b54ad8f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:16.891658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6acd6-189f-4c8a-9735-6851b54ad8f9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:16.893228 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:16.899446 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d94a3-f954-4469-aa8c-cc4f18cb16e3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:17.003580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d94a3-f954-4469-aa8c-cc4f18cb16e3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:17.029368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:17.040081 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446242a-e48e-44de-9c9e-8e197d88369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:17.040689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446242a-e48e-44de-9c9e-8e197d88369b None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:17.040973 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446242a-e48e-44de-9c9e-8e197d88369b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:17.041247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446242a-e48e-44de-9c9e-8e197d88369b None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:17.041896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446242a-e48e-44de-9c9e-8e197d88369b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:17.043392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446242a-e48e-44de-9c9e-8e197d88369b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:17.073909 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 28/54] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:34:16 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:20.339205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef54d9-b8bb-495e-b01f-4b82a7f03fa2 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:20.339721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef54d9-b8bb-495e-b01f-4b82a7f03fa2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:20.339793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef54d9-b8bb-495e-b01f-4b82a7f03fa2 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:20.341107 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 27/55] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:20.348861 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2c69-d156-48a6-9d0e-8655ed7319ff None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:20.349338 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2c69-d156-48a6-9d0e-8655ed7319ff None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:20.349412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2c69-d156-48a6-9d0e-8655ed7319ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:20.530170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3e4c2c69-d156-48a6-9d0e-8655ed7319ff None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:20.531417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3e4c2c69-d156-48a6-9d0e-8655ed7319ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:20.537791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836813-c0a9-4779-862a-045523361e3a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:20.538321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836813-c0a9-4779-862a-045523361e3a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:20.538810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836813-c0a9-4779-862a-045523361e3a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:20.540609 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 28/56] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:20.546812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d1e22-3ec5-47da-a529-7eec49557362 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:20.547317 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d1e22-3ec5-47da-a529-7eec49557362 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:20.547824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d1e22-3ec5-47da-a529-7eec49557362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:20.580339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ea3d1e22-3ec5-47da-a529-7eec49557362 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:20.581789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ea3d1e22-3ec5-47da-a529-7eec49557362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:20.617278 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 29/57] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:20.618789 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecf361-c855-4df8-b21e-0d13504943ee None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:20.619217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecf361-c855-4df8-b21e-0d13504943ee None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:20.619659 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecf361-c855-4df8-b21e-0d13504943ee None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:20.621034 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 30/58] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:20.640319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e79634-17f2-4c05-82e0-23cc91d27011 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:20.640824 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e79634-17f2-4c05-82e0-23cc91d27011 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:20.641255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e79634-17f2-4c05-82e0-23cc91d27011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:20.660575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-22e79634-17f2-4c05-82e0-23cc91d27011 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:20.662472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-22e79634-17f2-4c05-82e0-23cc91d27011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:20.684977 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 29/59] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:20.732360 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 31/60] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:20.905236 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d42f46-10d0-4453-a346-1a80d43d8079 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:20.905793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d42f46-10d0-4453-a346-1a80d43d8079 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:20.906275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d42f46-10d0-4453-a346-1a80d43d8079 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:20.907887 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 30/61] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:20.914052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8231f-ea0a-4722-8bfb-dcd8944ca87f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:20.914583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8231f-ea0a-4722-8bfb-dcd8944ca87f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:20.915095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8231f-ea0a-4722-8bfb-dcd8944ca87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:20.943727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0aa8231f-ea0a-4722-8bfb-dcd8944ca87f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:20.945274 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0aa8231f-ea0a-4722-8bfb-dcd8944ca87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:20.985579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:21.020254 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 32/62] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:21.030694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d42f46-10d0-4453-a346-1a80d43d8079 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.072752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8231f-ea0a-4722-8bfb-dcd8944ca87f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.096378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:21.169121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d42f46-10d0-4453-a346-1a80d43d8079 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.185293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8231f-ea0a-4722-8bfb-dcd8944ca87f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.208224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c50217-b49c-483a-8dca-1a94bd2394d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.208879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c50217-b49c-483a-8dca-1a94bd2394d5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.209162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c50217-b49c-483a-8dca-1a94bd2394d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.209439 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c50217-b49c-483a-8dca-1a94bd2394d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.220487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c50217-b49c-483a-8dca-1a94bd2394d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:21.220791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc4cc6-99d2-40b1-a91c-fff6eacfde86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.221426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc4cc6-99d2-40b1-a91c-fff6eacfde86 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.221710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc4cc6-99d2-40b1-a91c-fff6eacfde86 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.221989 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc4cc6-99d2-40b1-a91c-fff6eacfde86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-88c50217-b49c-483a-8dca-1a94bd2394d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:21.225297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:21.226114 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 31/63] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:21.227673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c50217-b49c-483a-8dca-1a94bd2394d5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.242099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc4cc6-99d2-40b1-a91c-fff6eacfde86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-29bc4cc6-99d2-40b1-a91c-fff6eacfde86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:21.249051 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:21.249923 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:21.251213 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 33/64] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:21.251853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfa35f-71df-422c-b1ae-c5afc1f13eaa None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.252122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfa35f-71df-422c-b1ae-c5afc1f13eaa None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.252287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfa35f-71df-422c-b1ae-c5afc1f13eaa None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.253640 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 34/65] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:21.254237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfa35f-71df-422c-b1ae-c5afc1f13eaa None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.292725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:21.338164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:21.342545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c50217-b49c-483a-8dca-1a94bd2394d5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.363269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfa35f-71df-422c-b1ae-c5afc1f13eaa None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.376226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830dc769-14b4-46ba-898d-3250e3358623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.376860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830dc769-14b4-46ba-898d-3250e3358623 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.377140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830dc769-14b4-46ba-898d-3250e3358623 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.377417 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830dc769-14b4-46ba-898d-3250e3358623 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.388401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830dc769-14b4-46ba-898d-3250e3358623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-830dc769-14b4-46ba-898d-3250e3358623 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:21.393164 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:21.394016 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:21.394815 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 32/66] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:21.395456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830dc769-14b4-46ba-898d-3250e3358623 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.400327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe666351-1d92-4855-be49-415a11c66db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.400941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe666351-1d92-4855-be49-415a11c66db5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.401223 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe666351-1d92-4855-be49-415a11c66db5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.401500 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe666351-1d92-4855-be49-415a11c66db5 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.402172 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe666351-1d92-4855-be49-415a11c66db5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:21.402478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe666351-1d92-4855-be49-415a11c66db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:21.403984 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe666351-1d92-4855-be49-415a11c66db5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:21.416137 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 35/67] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:21.417316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619e8b8-b84a-44a0-ac98-8d336497e6a8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.417604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619e8b8-b84a-44a0-ac98-8d336497e6a8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.417891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619e8b8-b84a-44a0-ac98-8d336497e6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.435404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8619e8b8-b84a-44a0-ac98-8d336497e6a8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:21.436540 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8619e8b8-b84a-44a0-ac98-8d336497e6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:21.497461 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830dc769-14b4-46ba-898d-3250e3358623 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.502068 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 36/68] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:21.502937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619e8b8-b84a-44a0-ac98-8d336497e6a8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.534580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c443-e509-4a1a-9d43-ef00c91f8b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.534785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:21.535377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c443-e509-4a1a-9d43-ef00c91f8b8c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.535655 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c443-e509-4a1a-9d43-ef00c91f8b8c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.535926 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c443-e509-4a1a-9d43-ef00c91f8b8c None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.536634 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3c443-e509-4a1a-9d43-ef00c91f8b8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:21.537291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3c443-e509-4a1a-9d43-ef00c91f8b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:21.538687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3c443-e509-4a1a-9d43-ef00c91f8b8c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:21.551231 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 33/69] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:21.551231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c443-e509-4a1a-9d43-ef00c91f8b8c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.569707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:21.625885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619e8b8-b84a-44a0-ac98-8d336497e6a8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.642995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c443-e509-4a1a-9d43-ef00c91f8b8c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.665383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f50322-c379-49ea-bee5-051d02e0d9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.666328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f50322-c379-49ea-bee5-051d02e0d9d5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.666806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f50322-c379-49ea-bee5-051d02e0d9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.667249 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f50322-c379-49ea-bee5-051d02e0d9d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.675717 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15e26f-62cf-441c-8a49-6129b9e44e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.676337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15e26f-62cf-441c-8a49-6129b9e44e36 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.676616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15e26f-62cf-441c-8a49-6129b9e44e36 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.676887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15e26f-62cf-441c-8a49-6129b9e44e36 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.677545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15e26f-62cf-441c-8a49-6129b9e44e36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:21.677845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15e26f-62cf-441c-8a49-6129b9e44e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:21.679273 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15e26f-62cf-441c-8a49-6129b9e44e36 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:21.679585 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f50322-c379-49ea-bee5-051d02e0d9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c3f50322-c379-49ea-bee5-051d02e0d9d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:21.684983 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:21.685906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:21.686738 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:21.688834 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 37/70] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:21.689753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f50322-c379-49ea-bee5-051d02e0d9d5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.692192 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 34/71] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:21.693288 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15e26f-62cf-441c-8a49-6129b9e44e36 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.774970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:21.779537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:21.794053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15e26f-62cf-441c-8a49-6129b9e44e36 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.795863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f50322-c379-49ea-bee5-051d02e0d9d5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.827152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55965c7e-0086-4817-83d2-956b3df6beb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.827797 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55965c7e-0086-4817-83d2-956b3df6beb0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.828073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55965c7e-0086-4817-83d2-956b3df6beb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.828348 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55965c7e-0086-4817-83d2-956b3df6beb0 None admin] PATH_INFO: `/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.829320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f698018-9178-4a15-936c-97ad532105e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.829467 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55965c7e-0086-4817-83d2-956b3df6beb0 None admin] RBAC: Authorizing `identity:get_user(user_id=2e8c1f674ea94c1eb2d8a47d1aad00af)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:21.829947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f698018-9178-4a15-936c-97ad532105e8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.830231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f698018-9178-4a15-936c-97ad532105e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.830505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f698018-9178-4a15-936c-97ad532105e8 None admin] PATH_INFO: `/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.830966 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55965c7e-0086-4817-83d2-956b3df6beb0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:21.832539 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 35/72] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:21.833296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55965c7e-0086-4817-83d2-956b3df6beb0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.849979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f698018-9178-4a15-936c-97ad532105e8 None admin] RBAC: Authorizing `identity:get_user(user_id=5acfe6e3e7be4573aab3cde32be1d7ea)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:21.851507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f698018-9178-4a15-936c-97ad532105e8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:21.853179 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 38/73] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:21.853936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f698018-9178-4a15-936c-97ad532105e8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.929948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55965c7e-0086-4817-83d2-956b3df6beb0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.954230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f698018-9178-4a15-936c-97ad532105e8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:21.963151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19a1fa-a534-4fd6-8ab6-343100f33e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.963791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19a1fa-a534-4fd6-8ab6-343100f33e48 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.964075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19a1fa-a534-4fd6-8ab6-343100f33e48 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.964354 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19a1fa-a534-4fd6-8ab6-343100f33e48 None admin] PATH_INFO: `/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.965513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19a1fa-a534-4fd6-8ab6-343100f33e48 None admin] RBAC: Authorizing `identity:get_user(user_id=5acfe6e3e7be4573aab3cde32be1d7ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:21.967011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19a1fa-a534-4fd6-8ab6-343100f33e48 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:21.968604 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 36/74] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:21.969391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19a1fa-a534-4fd6-8ab6-343100f33e48 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:21.988202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:21.993460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67312505-dd5c-4bbf-9315-c18ae0ccac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:21.994078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67312505-dd5c-4bbf-9315-c18ae0ccac5c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:21.994345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67312505-dd5c-4bbf-9315-c18ae0ccac5c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:21.994815 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67312505-dd5c-4bbf-9315-c18ae0ccac5c None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:21.995490 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67312505-dd5c-4bbf-9315-c18ae0ccac5c None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:21.995796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67312505-dd5c-4bbf-9315-c18ae0ccac5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:21.997201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67312505-dd5c-4bbf-9315-c18ae0ccac5c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.005351 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 39/75] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:22.006153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67312505-dd5c-4bbf-9315-c18ae0ccac5c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.011047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:22.068689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19a1fa-a534-4fd6-8ab6-343100f33e48 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.100557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b16e95-2cee-418b-9d0a-7d864f3327c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.101203 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b16e95-2cee-418b-9d0a-7d864f3327c0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.101488 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b16e95-2cee-418b-9d0a-7d864f3327c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.101768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b16e95-2cee-418b-9d0a-7d864f3327c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.102910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b16e95-2cee-418b-9d0a-7d864f3327c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.104389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b16e95-2cee-418b-9d0a-7d864f3327c0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.105966 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 37/76] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:22.106739 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b16e95-2cee-418b-9d0a-7d864f3327c0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.107097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67312505-dd5c-4bbf-9315-c18ae0ccac5c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.142012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218e0d-0f10-4a30-a470-91eb8fa57adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.142644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218e0d-0f10-4a30-a470-91eb8fa57adf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.142959 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218e0d-0f10-4a30-a470-91eb8fa57adf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.143238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218e0d-0f10-4a30-a470-91eb8fa57adf None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.144372 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e218e0d-0f10-4a30-a470-91eb8fa57adf None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.145820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e218e0d-0f10-4a30-a470-91eb8fa57adf None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.147660 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 40/77] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:22 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:22.148235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218e0d-0f10-4a30-a470-91eb8fa57adf None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.195449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:22.210060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b16e95-2cee-418b-9d0a-7d864f3327c0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.217269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:22.242311 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598bbe0-3b6b-4cbe-b9de-dfb64b3038da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.242981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598bbe0-3b6b-4cbe-b9de-dfb64b3038da None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.243266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598bbe0-3b6b-4cbe-b9de-dfb64b3038da None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.243560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598bbe0-3b6b-4cbe-b9de-dfb64b3038da None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.244700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598bbe0-3b6b-4cbe-b9de-dfb64b3038da None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.246158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598bbe0-3b6b-4cbe-b9de-dfb64b3038da None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.246256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218e0d-0f10-4a30-a470-91eb8fa57adf None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.247909 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 38/78] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:22 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:22.248756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598bbe0-3b6b-4cbe-b9de-dfb64b3038da None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.286677 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0c70f-af15-400f-99b6-bc32e776b103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.287703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0c70f-af15-400f-99b6-bc32e776b103 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.287985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0c70f-af15-400f-99b6-bc32e776b103 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.288260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0c70f-af15-400f-99b6-bc32e776b103 None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.289382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0c70f-af15-400f-99b6-bc32e776b103 None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.290894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0c70f-af15-400f-99b6-bc32e776b103 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.292540 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 41/79] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:22 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:22.293275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0c70f-af15-400f-99b6-bc32e776b103 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.350927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598bbe0-3b6b-4cbe-b9de-dfb64b3038da None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.382766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fd03c-dfdf-43ff-8568-02a902987968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.383414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fd03c-dfdf-43ff-8568-02a902987968 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.383691 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fd03c-dfdf-43ff-8568-02a902987968 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.383964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fd03c-dfdf-43ff-8568-02a902987968 None admin] PATH_INFO: `/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.385385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fd03c-dfdf-43ff-8568-02a902987968 None admin] RBAC: Authorizing `identity:get_user(user_id=5acfe6e3e7be4573aab3cde32be1d7ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.386863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fd03c-dfdf-43ff-8568-02a902987968 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.387942 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0c70f-af15-400f-99b6-bc32e776b103 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.388710 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 39/80] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:22 2022] GET /identity/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:22.389277 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fd03c-dfdf-43ff-8568-02a902987968 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.414259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:22.425416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34443b6-bf85-4ea3-a8d7-8f89c6175b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.426100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34443b6-bf85-4ea3-a8d7-8f89c6175b15 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.426389 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34443b6-bf85-4ea3-a8d7-8f89c6175b15 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.426663 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34443b6-bf85-4ea3-a8d7-8f89c6175b15 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.427348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34443b6-bf85-4ea3-a8d7-8f89c6175b15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.428785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34443b6-bf85-4ea3-a8d7-8f89c6175b15 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.439803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:22.451464 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 42/81] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:34:22 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:22.467105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34443b6-bf85-4ea3-a8d7-8f89c6175b15 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.494182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fd03c-dfdf-43ff-8568-02a902987968 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.535686 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced68fd0-22e1-4256-82e7-64d843c2f630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.536797 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced68fd0-22e1-4256-82e7-64d843c2f630 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.537451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced68fd0-22e1-4256-82e7-64d843c2f630 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.537738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced68fd0-22e1-4256-82e7-64d843c2f630 None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/users/2e8c1f674ea94c1eb2d8a47d1aad00af/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.540138 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced68fd0-22e1-4256-82e7-64d843c2f630 None admin] RBAC: Authorizing `identity:create_grant(project_id=538dc9f6865b4b1a932b75e3be851bb7, user_id=2e8c1f674ea94c1eb2d8a47d1aad00af, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.541646 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced68fd0-22e1-4256-82e7-64d843c2f630 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.570747 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 40/82] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:34:22 2022] PUT /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/users/2e8c1f674ea94c1eb2d8a47d1aad00af/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:22.570747 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced68fd0-22e1-4256-82e7-64d843c2f630 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.587314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34443b6-bf85-4ea3-a8d7-8f89c6175b15 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.628678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d90a3-f944-4a73-8cd2-78ccfe40f998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.629481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d90a3-f944-4a73-8cd2-78ccfe40f998 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.629880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d90a3-f944-4a73-8cd2-78ccfe40f998 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.630227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d90a3-f944-4a73-8cd2-78ccfe40f998 None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/users/5acfe6e3e7be4573aab3cde32be1d7ea/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.632594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d90a3-f944-4a73-8cd2-78ccfe40f998 None admin] RBAC: Authorizing `identity:create_grant(project_id=538dc9f6865b4b1a932b75e3be851bb7, user_id=5acfe6e3e7be4573aab3cde32be1d7ea, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.634391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d90a3-f944-4a73-8cd2-78ccfe40f998 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.657201 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 43/83] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:34:22 2022] PUT /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/users/5acfe6e3e7be4573aab3cde32be1d7ea/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:22.662961 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d90a3-f944-4a73-8cd2-78ccfe40f998 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.692173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:22.725235 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced68fd0-22e1-4256-82e7-64d843c2f630 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.745512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:22.769587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff91e1f-e454-44e5-8336-dcac858ae040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.770281 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff91e1f-e454-44e5-8336-dcac858ae040 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.770564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff91e1f-e454-44e5-8336-dcac858ae040 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.770883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff91e1f-e454-44e5-8336-dcac858ae040 None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/users/5acfe6e3e7be4573aab3cde32be1d7ea/roles/5cd206a783e44ac2829c7661b34bb6a7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.773490 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff91e1f-e454-44e5-8336-dcac858ae040 None admin] RBAC: Authorizing `identity:create_grant(project_id=538dc9f6865b4b1a932b75e3be851bb7, user_id=5acfe6e3e7be4573aab3cde32be1d7ea, role_id=5cd206a783e44ac2829c7661b34bb6a7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.775036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff91e1f-e454-44e5-8336-dcac858ae040 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.795480 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 41/84] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:34:22 2022] PUT /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/users/5acfe6e3e7be4573aab3cde32be1d7ea/roles/5cd206a783e44ac2829c7661b34bb6a7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:22.804091 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d90a3-f944-4a73-8cd2-78ccfe40f998 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:22.846695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a149d-5328-477e-86b8-9e651b5220e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:22.847408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a149d-5328-477e-86b8-9e651b5220e1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:22.847692 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a149d-5328-477e-86b8-9e651b5220e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:22.847972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a149d-5328-477e-86b8-9e651b5220e1 None admin] PATH_INFO: `/v3/projects/f0388fccf3c54977b8da1e55ff5bdc24` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:22.849376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a149d-5328-477e-86b8-9e651b5220e1 None admin] RBAC: Authorizing `identity:get_project(project_id=f0388fccf3c54977b8da1e55ff5bdc24)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:22.851210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a149d-5328-477e-86b8-9e651b5220e1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:22.853021 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 44/85] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:22 2022] GET /identity/v3/projects/f0388fccf3c54977b8da1e55ff5bdc24 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:22.859326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff91e1f-e454-44e5-8336-dcac858ae040 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:22.985368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff91e1f-e454-44e5-8336-dcac858ae040 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:23.023860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6eb51-b77c-40b5-8264-5bf77d7656c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:23.024545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6eb51-b77c-40b5-8264-5bf77d7656c3 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:23.024879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6eb51-b77c-40b5-8264-5bf77d7656c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:23.025158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6eb51-b77c-40b5-8264-5bf77d7656c3 None admin] PATH_INFO: `/v3/projects/f0388fccf3c54977b8da1e55ff5bdc24/users/5acfe6e3e7be4573aab3cde32be1d7ea/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:23.027158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6eb51-b77c-40b5-8264-5bf77d7656c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0388fccf3c54977b8da1e55ff5bdc24, user_id=5acfe6e3e7be4573aab3cde32be1d7ea, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:23.028776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6eb51-b77c-40b5-8264-5bf77d7656c3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:23.046746 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 42/86] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:34:22 2022] PUT /identity/v3/projects/f0388fccf3c54977b8da1e55ff5bdc24/users/5acfe6e3e7be4573aab3cde32be1d7ea/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:25.218147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d32e37-b0df-4e4f-a7af-2121021f462d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:25.218671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d32e37-b0df-4e4f-a7af-2121021f462d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:25.218806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d32e37-b0df-4e4f-a7af-2121021f462d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:25.220075 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 45/87] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:25.225571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b10ee2-31a4-4603-9484-ad4c9199b190 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:25.225974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b10ee2-31a4-4603-9484-ad4c9199b190 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:25.226116 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b10ee2-31a4-4603-9484-ad4c9199b190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:25.237556 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:25.252042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b1b10ee2-31a4-4603-9484-ad4c9199b190 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:25.253918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b1b10ee2-31a4-4603-9484-ad4c9199b190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:25.360965 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 43/88] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:25.582298 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0cd33-4659-43de-8c9c-1d5524ecfc84 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:25.582981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0cd33-4659-43de-8c9c-1d5524ecfc84 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:25.583486 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0cd33-4659-43de-8c9c-1d5524ecfc84 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:25.585065 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 46/89] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:25.590448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beaa2e-a94d-423a-8202-90f94d68ccd4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:25.590752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beaa2e-a94d-423a-8202-90f94d68ccd4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:25.591034 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7beaa2e-a94d-423a-8202-90f94d68ccd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:25.615705 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a7beaa2e-a94d-423a-8202-90f94d68ccd4 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:25.616893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a7beaa2e-a94d-423a-8202-90f94d68ccd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:25.699408 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 44/90] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:25.807690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0cd33-4659-43de-8c9c-1d5524ecfc84 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:25.932625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0cd33-4659-43de-8c9c-1d5524ecfc84 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:25.967829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33965f-6b89-4571-a496-f854bf23c5d3 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:25.968159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33965f-6b89-4571-a496-f854bf23c5d3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:25.968391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33965f-6b89-4571-a496-f854bf23c5d3 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:25.970190 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 45/91] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:25.971512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:25.980248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09398a59-4f39-457b-871f-01f7d9e10b8c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:25.980805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09398a59-4f39-457b-871f-01f7d9e10b8c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:25.981177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09398a59-4f39-457b-871f-01f7d9e10b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:25.986662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fba77b-d13e-469d-a544-a5f4ebf783a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:25.988395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fba77b-d13e-469d-a544-a5f4ebf783a1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:25.988671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fba77b-d13e-469d-a544-a5f4ebf783a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:25.988950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fba77b-d13e-469d-a544-a5f4ebf783a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.012433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fba77b-d13e-469d-a544-a5f4ebf783a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-79fba77b-d13e-469d-a544-a5f4ebf783a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:26.017267 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:26.018130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:26.018926 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 47/92] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:34:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:26.020044 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-09398a59-4f39-457b-871f-01f7d9e10b8c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:26.020795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-09398a59-4f39-457b-871f-01f7d9e10b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:26.032526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fba77b-d13e-469d-a544-a5f4ebf783a1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:26.127895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:26.143325 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 46/93] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:26.144804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c3325-209b-4e9b-82eb-000dfcd2c26c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.145219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c3325-209b-4e9b-82eb-000dfcd2c26c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.145618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c3325-209b-4e9b-82eb-000dfcd2c26c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.151565 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 47/94] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:26.152478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c3325-209b-4e9b-82eb-000dfcd2c26c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:26.209489 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fba77b-d13e-469d-a544-a5f4ebf783a1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:26.296030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a579d24-fdfa-463d-bf60-7c2bd5f26453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:26.296697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a579d24-fdfa-463d-bf60-7c2bd5f26453 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.296985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a579d24-fdfa-463d-bf60-7c2bd5f26453 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.297261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a579d24-fdfa-463d-bf60-7c2bd5f26453 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.297938 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a579d24-fdfa-463d-bf60-7c2bd5f26453 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:26.298245 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a579d24-fdfa-463d-bf60-7c2bd5f26453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:26.299744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a579d24-fdfa-463d-bf60-7c2bd5f26453 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:26.318350 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 48/95] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:26.322689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3c39d-2ce5-45a1-885d-f96547b9f586 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.323217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3c39d-2ce5-45a1-885d-f96547b9f586 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.325018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3c39d-2ce5-45a1-885d-f96547b9f586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.348499 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c3325-209b-4e9b-82eb-000dfcd2c26c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:26.366607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d4f3c39d-2ce5-45a1-885d-f96547b9f586 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:26.367897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d4f3c39d-2ce5-45a1-885d-f96547b9f586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:26.397662 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7d6aa-5296-4e57-b423-3b63eb27dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:26.398544 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7d6aa-5296-4e57-b423-3b63eb27dbae None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.398992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7d6aa-5296-4e57-b423-3b63eb27dbae None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.399404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7d6aa-5296-4e57-b423-3b63eb27dbae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.411621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7d6aa-5296-4e57-b423-3b63eb27dbae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-57c7d6aa-5296-4e57-b423-3b63eb27dbae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:26.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:26.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:26.420410 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:26.422917 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 48/96] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:34:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:26.424474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:26.424698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7d6aa-5296-4e57-b423-3b63eb27dbae None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:26.452122 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 49/97] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:26.453584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a886d65-1e43-4be6-82ad-ad7d9953af3a None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.453870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a886d65-1e43-4be6-82ad-ad7d9953af3a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.454149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a886d65-1e43-4be6-82ad-ad7d9953af3a None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.455424 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 50/98] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:26.458860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a886d65-1e43-4be6-82ad-ad7d9953af3a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:26.491883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:26.543995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7d6aa-5296-4e57-b423-3b63eb27dbae None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:26.559315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a886d65-1e43-4be6-82ad-ad7d9953af3a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:26.581612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23818c47-414b-4f9d-adad-bbf3e2a6c51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:26.582673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23818c47-414b-4f9d-adad-bbf3e2a6c51a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.583190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23818c47-414b-4f9d-adad-bbf3e2a6c51a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.583743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23818c47-414b-4f9d-adad-bbf3e2a6c51a None admin] PATH_INFO: `/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.585354 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23818c47-414b-4f9d-adad-bbf3e2a6c51a None admin] RBAC: Authorizing `identity:get_user(user_id=2e8c1f674ea94c1eb2d8a47d1aad00af)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:26.587329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23818c47-414b-4f9d-adad-bbf3e2a6c51a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:26.589522 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 49/99] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:26 2022] GET /identity/v3/users/2e8c1f674ea94c1eb2d8a47d1aad00af => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:26.590915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23818c47-414b-4f9d-adad-bbf3e2a6c51a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:26.602739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113f0a7-8b21-4014-852e-7daf58bfe65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:26.605832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113f0a7-8b21-4014-852e-7daf58bfe65f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.605832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113f0a7-8b21-4014-852e-7daf58bfe65f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.605832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113f0a7-8b21-4014-852e-7daf58bfe65f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.618846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113f0a7-8b21-4014-852e-7daf58bfe65f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c113f0a7-8b21-4014-852e-7daf58bfe65f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:26.624974 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:26.625970 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:26.627530 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 51/100] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:26.628352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3282c7-20cd-4f12-a3b9-41bfdfc12199 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.628352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3282c7-20cd-4f12-a3b9-41bfdfc12199 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.628352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3282c7-20cd-4f12-a3b9-41bfdfc12199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.653829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-dd3282c7-20cd-4f12-a3b9-41bfdfc12199 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:26.655340 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dd3282c7-20cd-4f12-a3b9-41bfdfc12199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:26.691482 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:26.713520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23818c47-414b-4f9d-adad-bbf3e2a6c51a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:26.734761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:26.735305 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 52/101] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:26.736148 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3282c7-20cd-4f12-a3b9-41bfdfc12199 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:26.750605 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0ba5a-dcc5-4407-801e-97320a094225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:26.751193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0ba5a-dcc5-4407-801e-97320a094225 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.751479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0ba5a-dcc5-4407-801e-97320a094225 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.751754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0ba5a-dcc5-4407-801e-97320a094225 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.752426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0ba5a-dcc5-4407-801e-97320a094225 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:26.752743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0ba5a-dcc5-4407-801e-97320a094225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:26.754129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0ba5a-dcc5-4407-801e-97320a094225 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:26.764934 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 50/102] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:34:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:26.765998 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0ba5a-dcc5-4407-801e-97320a094225 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:26.843163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3282c7-20cd-4f12-a3b9-41bfdfc12199 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:26.874045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0ba5a-dcc5-4407-801e-97320a094225 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:26.878850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f259c6-6b57-46f4-a188-09a1f1e8fa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:26.879471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f259c6-6b57-46f4-a188-09a1f1e8fa6b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.879753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f259c6-6b57-46f4-a188-09a1f1e8fa6b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.880033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f259c6-6b57-46f4-a188-09a1f1e8fa6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.908059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529781e-6447-4aae-91c8-ec01794bc903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:26.908674 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529781e-6447-4aae-91c8-ec01794bc903 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:26.908958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529781e-6447-4aae-91c8-ec01794bc903 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:26.909234 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529781e-6447-4aae-91c8-ec01794bc903 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:26.909896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529781e-6447-4aae-91c8-ec01794bc903 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:26.910199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529781e-6447-4aae-91c8-ec01794bc903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:26.911611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529781e-6447-4aae-91c8-ec01794bc903 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:26.915077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:26.920895 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 51/103] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:26.921701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529781e-6447-4aae-91c8-ec01794bc903 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:26.958024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f259c6-6b57-46f4-a188-09a1f1e8fa6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:26.959551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f259c6-6b57-46f4-a188-09a1f1e8fa6b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:26.975632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:27.031114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529781e-6447-4aae-91c8-ec01794bc903 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.061728 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 53/104] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:27.063239 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d0671-6400-4ffe-b2c8-b297888839b0 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.063520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d0671-6400-4ffe-b2c8-b297888839b0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.063795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d0671-6400-4ffe-b2c8-b297888839b0 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.065032 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 54/105] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:27.065716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d0671-6400-4ffe-b2c8-b297888839b0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.068421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a41a7-4e53-491c-865a-8a765c6486fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.069351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a41a7-4e53-491c-865a-8a765c6486fe None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.069630 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a41a7-4e53-491c-865a-8a765c6486fe None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.069907 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a41a7-4e53-491c-865a-8a765c6486fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.080510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a41a7-4e53-491c-865a-8a765c6486fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-c21a41a7-4e53-491c-865a-8a765c6486fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:27.085784 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:27.086849 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 52/106] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:27.091625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a41a7-4e53-491c-865a-8a765c6486fe None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.136531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:27.172290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d0671-6400-4ffe-b2c8-b297888839b0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.184513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a41a7-4e53-491c-865a-8a765c6486fe None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.186791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:27.205747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230134-710f-45ac-b7da-72f010b15d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.206354 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230134-710f-45ac-b7da-72f010b15d6e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.206630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230134-710f-45ac-b7da-72f010b15d6e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.206936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230134-710f-45ac-b7da-72f010b15d6e None admin] PATH_INFO: `/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.208456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86230134-710f-45ac-b7da-72f010b15d6e None admin] RBAC: Authorizing `identity:get_user(user_id=5acfe6e3e7be4573aab3cde32be1d7ea)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:27.209713 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86230134-710f-45ac-b7da-72f010b15d6e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:27.211329 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 55/107] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:27 2022] GET /identity/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:27.212083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230134-710f-45ac-b7da-72f010b15d6e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.221885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b739485-d5ab-4e7c-affa-616acba44f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.222529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b739485-d5ab-4e7c-affa-616acba44f3c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.222847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b739485-d5ab-4e7c-affa-616acba44f3c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.223127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b739485-d5ab-4e7c-affa-616acba44f3c None admin] PATH_INFO: `/v3/users/a59abf99e567496e82501c2b0cc22f6f` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.236851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b739485-d5ab-4e7c-affa-616acba44f3c None admin] RBAC: Authorizing `identity:get_user(user_id=a59abf99e567496e82501c2b0cc22f6f)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:27.238334 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b739485-d5ab-4e7c-affa-616acba44f3c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:27.240093 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 53/108] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:27 2022] GET /identity/v3/users/a59abf99e567496e82501c2b0cc22f6f => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:27.240846 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b739485-d5ab-4e7c-affa-616acba44f3c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.316528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230134-710f-45ac-b7da-72f010b15d6e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.338025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b739485-d5ab-4e7c-affa-616acba44f3c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.350036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7047c9-45d0-4b35-9675-81d84f64f086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.350676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7047c9-45d0-4b35-9675-81d84f64f086 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.350974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7047c9-45d0-4b35-9675-81d84f64f086 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.351242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7047c9-45d0-4b35-9675-81d84f64f086 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.352349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7047c9-45d0-4b35-9675-81d84f64f086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:27.353785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7047c9-45d0-4b35-9675-81d84f64f086 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:27.355366 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 56/109] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:27.356433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd1bad-dc57-4103-9d6f-177929297b75 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.356785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd1bad-dc57-4103-9d6f-177929297b75 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.357054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd1bad-dc57-4103-9d6f-177929297b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.363416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:27.373206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90168-44b9-4fb9-a113-42b3fd0700f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.373854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90168-44b9-4fb9-a113-42b3fd0700f6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.374137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90168-44b9-4fb9-a113-42b3fd0700f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.374413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90168-44b9-4fb9-a113-42b3fd0700f6 None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.375580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90168-44b9-4fb9-a113-42b3fd0700f6 None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:27.375849 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-60cd1bad-dc57-4103-9d6f-177929297b75 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:27.377035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90168-44b9-4fb9-a113-42b3fd0700f6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:27.377129 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-60cd1bad-dc57-4103-9d6f-177929297b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:27.378665 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 54/110] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:27 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:27.379449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90168-44b9-4fb9-a113-42b3fd0700f6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.407598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:27.442381 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 57/111] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:27.448195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd1bad-dc57-4103-9d6f-177929297b75 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.497058 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90168-44b9-4fb9-a113-42b3fd0700f6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.537394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95c8e5-4a3d-4651-9d9f-78aef3c33f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.537932 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95c8e5-4a3d-4651-9d9f-78aef3c33f68 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.538215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95c8e5-4a3d-4651-9d9f-78aef3c33f68 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.538490 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95c8e5-4a3d-4651-9d9f-78aef3c33f68 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.539234 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95c8e5-4a3d-4651-9d9f-78aef3c33f68 None admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:27.539497 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95c8e5-4a3d-4651-9d9f-78aef3c33f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:27.540888 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95c8e5-4a3d-4651-9d9f-78aef3c33f68 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:27.545926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd1bad-dc57-4103-9d6f-177929297b75 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.549573 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 55/112] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:27.550374 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95c8e5-4a3d-4651-9d9f-78aef3c33f68 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.576280 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:27.582346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea98bfa-fad7-40d3-97a6-9fafcba6092b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.583134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea98bfa-fad7-40d3-97a6-9fafcba6092b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.583813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea98bfa-fad7-40d3-97a6-9fafcba6092b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.584096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea98bfa-fad7-40d3-97a6-9fafcba6092b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.645268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:27.654749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95c8e5-4a3d-4651-9d9f-78aef3c33f68 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.655783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea98bfa-fad7-40d3-97a6-9fafcba6092b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:27.657269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea98bfa-fad7-40d3-97a6-9fafcba6092b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:27.688514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62465b0f-15b4-4d54-a91e-078d54577076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.689152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62465b0f-15b4-4d54-a91e-078d54577076 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.689440 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62465b0f-15b4-4d54-a91e-078d54577076 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.689768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62465b0f-15b4-4d54-a91e-078d54577076 None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.690874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62465b0f-15b4-4d54-a91e-078d54577076 None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:27.692332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62465b0f-15b4-4d54-a91e-078d54577076 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:27.698621 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 56/113] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:27 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:27.698621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62465b0f-15b4-4d54-a91e-078d54577076 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.763249 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 58/114] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:27.763249 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea98bfa-fad7-40d3-97a6-9fafcba6092b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.790843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:27.798620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62465b0f-15b4-4d54-a91e-078d54577076 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.831036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411262a-ef3c-4c0f-a08e-d86acf95c4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.831657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411262a-ef3c-4c0f-a08e-d86acf95c4e7 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.831942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411262a-ef3c-4c0f-a08e-d86acf95c4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.832218 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411262a-ef3c-4c0f-a08e-d86acf95c4e7 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.832873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411262a-ef3c-4c0f-a08e-d86acf95c4e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:27.834321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411262a-ef3c-4c0f-a08e-d86acf95c4e7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:27.860784 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea98bfa-fad7-40d3-97a6-9fafcba6092b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:27.863670 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 57/115] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:34:27 2022] POST /identity/v3/projects => generated 302 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:27.864491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411262a-ef3c-4c0f-a08e-d86acf95c4e7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:27.884841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:27.905451 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60736f43-172d-4791-8fe9-7ec067bbb728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:27.906381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60736f43-172d-4791-8fe9-7ec067bbb728 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:27.906668 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60736f43-172d-4791-8fe9-7ec067bbb728 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:27.906976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60736f43-172d-4791-8fe9-7ec067bbb728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:27.907730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60736f43-172d-4791-8fe9-7ec067bbb728 None admin] RBAC: Adding query filter params (role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.project.id=538dc9f6865b4b1a932b75e3be851bb7, user.id=2e8c1f674ea94c1eb2d8a47d1aad00af) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:27.908044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60736f43-172d-4791-8fe9-7ec067bbb728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:27.909547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60736f43-172d-4791-8fe9-7ec067bbb728 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:27.919086 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 59/116] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:34:27 2022] GET /identity/v3/role_assignments?user.id=2e8c1f674ea94c1eb2d8a47d1aad00af&scope.project.id=538dc9f6865b4b1a932b75e3be851bb7&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:27.920018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60736f43-172d-4791-8fe9-7ec067bbb728 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:28.006009 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411262a-ef3c-4c0f-a08e-d86acf95c4e7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:28.051089 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633f89d-f71b-4b2e-ae80-5836ac34ef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:28.052240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633f89d-f71b-4b2e-ae80-5836ac34ef3e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:28.052733 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633f89d-f71b-4b2e-ae80-5836ac34ef3e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:28.053262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633f89d-f71b-4b2e-ae80-5836ac34ef3e None admin] PATH_INFO: `/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:28.055009 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633f89d-f71b-4b2e-ae80-5836ac34ef3e None admin] RBAC: Authorizing `identity:get_user(user_id=5acfe6e3e7be4573aab3cde32be1d7ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:28.056772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633f89d-f71b-4b2e-ae80-5836ac34ef3e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:28.059001 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 58/117] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:27 2022] GET /identity/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:28.060153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633f89d-f71b-4b2e-ae80-5836ac34ef3e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:28.073219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:28.083407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60736f43-172d-4791-8fe9-7ec067bbb728 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:28.136341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4cb14-d395-40ea-a374-38be06f4763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:28.137027 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4cb14-d395-40ea-a374-38be06f4763b None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:28.137312 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4cb14-d395-40ea-a374-38be06f4763b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:28.137590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4cb14-d395-40ea-a374-38be06f4763b None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/users/a59abf99e567496e82501c2b0cc22f6f/roles/b96f3faefe1c453fb20deea4530c43f0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:28.140113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4cb14-d395-40ea-a374-38be06f4763b None admin] RBAC: Authorizing `identity:create_grant(project_id=538dc9f6865b4b1a932b75e3be851bb7, user_id=a59abf99e567496e82501c2b0cc22f6f, role_id=b96f3faefe1c453fb20deea4530c43f0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:28.141572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4cb14-d395-40ea-a374-38be06f4763b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:28.162591 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 60/118] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:34:27 2022] PUT /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/users/a59abf99e567496e82501c2b0cc22f6f/roles/b96f3faefe1c453fb20deea4530c43f0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:28.163579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4cb14-d395-40ea-a374-38be06f4763b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:28.164184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:28.187002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633f89d-f71b-4b2e-ae80-5836ac34ef3e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:28.239153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d6734-f204-458c-9e70-248c162c19d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:28.239840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d6734-f204-458c-9e70-248c162c19d4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:28.240121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d6734-f204-458c-9e70-248c162c19d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:28.240404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d6734-f204-458c-9e70-248c162c19d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:28.256830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d6734-f204-458c-9e70-248c162c19d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-9e1d6734-f204-458c-9e70-248c162c19d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:28.263649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:28.264736 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 59/119] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:28.265616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d6734-f204-458c-9e70-248c162c19d4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:28.312847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4cb14-d395-40ea-a374-38be06f4763b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:28.363757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aade8f6-3114-4b92-b0d3-ba6c007a717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:28.364400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aade8f6-3114-4b92-b0d3-ba6c007a717e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:28.364702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aade8f6-3114-4b92-b0d3-ba6c007a717e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:28.364987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aade8f6-3114-4b92-b0d3-ba6c007a717e None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:28.367169 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aade8f6-3114-4b92-b0d3-ba6c007a717e None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:28.368945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aade8f6-3114-4b92-b0d3-ba6c007a717e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:28.372001 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 61/120] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:28 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:28.372906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aade8f6-3114-4b92-b0d3-ba6c007a717e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:28.404475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:28.407001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d6734-f204-458c-9e70-248c162c19d4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:28.452094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa913b-10bf-43fc-9748-36522234d4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:28.452730 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa913b-10bf-43fc-9748-36522234d4f9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:28.453018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa913b-10bf-43fc-9748-36522234d4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:28.453299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa913b-10bf-43fc-9748-36522234d4f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:28.466648 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:28.484247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aade8f6-3114-4b92-b0d3-ba6c007a717e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:28.523927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82879605-9e37-4706-aca7-4691b853e1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:28.524949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82879605-9e37-4706-aca7-4691b853e1a9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:28.525399 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82879605-9e37-4706-aca7-4691b853e1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:28.525793 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82879605-9e37-4706-aca7-4691b853e1a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:28.526590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82879605-9e37-4706-aca7-4691b853e1a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:28.527292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82879605-9e37-4706-aca7-4691b853e1a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:28.529606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82879605-9e37-4706-aca7-4691b853e1a9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:28.538289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa913b-10bf-43fc-9748-36522234d4f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:28.539875 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa913b-10bf-43fc-9748-36522234d4f9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:28.540526 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 62/121] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:28.541610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82879605-9e37-4706-aca7-4691b853e1a9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:28.638851 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:28.652075 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 60/122] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:28.653250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa913b-10bf-43fc-9748-36522234d4f9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:28.657398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82879605-9e37-4706-aca7-4691b853e1a9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:28.697813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f2aa-d400-42b1-99bc-c15fa207b96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:28.699226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:28.700218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f2aa-d400-42b1-99bc-c15fa207b96d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:28.701152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f2aa-d400-42b1-99bc-c15fa207b96d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:28.701559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f2aa-d400-42b1-99bc-c15fa207b96d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:28.702674 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290f2aa-d400-42b1-99bc-c15fa207b96d None admin] RBAC: Adding query filter params (role.id=5cd206a783e44ac2829c7661b34bb6a7, scope.project.id=538dc9f6865b4b1a932b75e3be851bb7, user.id=5acfe6e3e7be4573aab3cde32be1d7ea) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:28.703141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290f2aa-d400-42b1-99bc-c15fa207b96d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:28.705074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290f2aa-d400-42b1-99bc-c15fa207b96d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:28.713171 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 63/123] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:34:28 2022] GET /identity/v3/role_assignments?user.id=5acfe6e3e7be4573aab3cde32be1d7ea&scope.project.id=538dc9f6865b4b1a932b75e3be851bb7&role.id=5cd206a783e44ac2829c7661b34bb6a7 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:28.718958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f2aa-d400-42b1-99bc-c15fa207b96d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:28.801324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa913b-10bf-43fc-9748-36522234d4f9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:28.873623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb1e93-7aab-4f15-b022-5f1db956b988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:28.874295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb1e93-7aab-4f15-b022-5f1db956b988 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:28.874579 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb1e93-7aab-4f15-b022-5f1db956b988 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:28.874890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb1e93-7aab-4f15-b022-5f1db956b988 None admin] PATH_INFO: `/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:28.876637 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb1e93-7aab-4f15-b022-5f1db956b988 None admin] RBAC: Authorizing `identity:get_user(user_id=5acfe6e3e7be4573aab3cde32be1d7ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:28.883792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb1e93-7aab-4f15-b022-5f1db956b988 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:28.886027 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 61/124] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:28 2022] GET /identity/v3/users/5acfe6e3e7be4573aab3cde32be1d7ea => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:28.891990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb1e93-7aab-4f15-b022-5f1db956b988 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:28.929664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f2aa-d400-42b1-99bc-c15fa207b96d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:28.972454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00c755-8711-4149-8e7a-88320d83a7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:28.973485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00c755-8711-4149-8e7a-88320d83a7aa None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:28.973937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00c755-8711-4149-8e7a-88320d83a7aa None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:28.974216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00c755-8711-4149-8e7a-88320d83a7aa None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:28.975575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00c755-8711-4149-8e7a-88320d83a7aa None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:28.977097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00c755-8711-4149-8e7a-88320d83a7aa None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:28.979579 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 64/125] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:28 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:28.985143 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00c755-8711-4149-8e7a-88320d83a7aa None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:29.010947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:29.039202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb1e93-7aab-4f15-b022-5f1db956b988 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:29.052134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:29.086292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab8159-c6f7-40cf-9ee8-601815bfc4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:29.087615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab8159-c6f7-40cf-9ee8-601815bfc4eb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:29.088137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab8159-c6f7-40cf-9ee8-601815bfc4eb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:29.088865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab8159-c6f7-40cf-9ee8-601815bfc4eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:29.108691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00c755-8711-4149-8e7a-88320d83a7aa None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:29.154528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97822dc7-5c61-40e7-bfa3-df21c8a4244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:29.155085 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97822dc7-5c61-40e7-bfa3-df21c8a4244d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:29.155364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97822dc7-5c61-40e7-bfa3-df21c8a4244d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:29.155637 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97822dc7-5c61-40e7-bfa3-df21c8a4244d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:29.156365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97822dc7-5c61-40e7-bfa3-df21c8a4244d None admin] RBAC: Adding query filter params (role.id=290368e3589e425e99a9f4dbfe2b3823, scope.project.id=538dc9f6865b4b1a932b75e3be851bb7, user.id=5acfe6e3e7be4573aab3cde32be1d7ea) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:29.156672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97822dc7-5c61-40e7-bfa3-df21c8a4244d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:29.158273 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97822dc7-5c61-40e7-bfa3-df21c8a4244d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:29.164770 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 65/126] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:34:28 2022] GET /identity/v3/role_assignments?user.id=5acfe6e3e7be4573aab3cde32be1d7ea&scope.project.id=538dc9f6865b4b1a932b75e3be851bb7&role.id=290368e3589e425e99a9f4dbfe2b3823 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:29.188076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab8159-c6f7-40cf-9ee8-601815bfc4eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:29.189933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab8159-c6f7-40cf-9ee8-601815bfc4eb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:29.300999 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 62/127] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 08:34:29.310740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97822dc7-5c61-40e7-bfa3-df21c8a4244d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:29.399046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:29.423143 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97822dc7-5c61-40e7-bfa3-df21c8a4244d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:29.456770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381961a4-d04a-449d-a325-316f1cb95d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:29.457413 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381961a4-d04a-449d-a325-316f1cb95d54 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:29.457695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381961a4-d04a-449d-a325-316f1cb95d54 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:29.457971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381961a4-d04a-449d-a325-316f1cb95d54 None admin] PATH_INFO: `/v3/projects/f0388fccf3c54977b8da1e55ff5bdc24` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:29.459113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381961a4-d04a-449d-a325-316f1cb95d54 None admin] RBAC: Authorizing `identity:get_project(project_id=f0388fccf3c54977b8da1e55ff5bdc24)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:29.460596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381961a4-d04a-449d-a325-316f1cb95d54 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:29.462519 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 66/128] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:29 2022] GET /identity/v3/projects/f0388fccf3c54977b8da1e55ff5bdc24 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:29.468253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab8159-c6f7-40cf-9ee8-601815bfc4eb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:29.478389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:29.569226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab8159-c6f7-40cf-9ee8-601815bfc4eb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:29.602912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d72e9a-885f-4ad8-a1af-2e77e9f7e65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:29.603533 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d72e9a-885f-4ad8-a1af-2e77e9f7e65b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:29.603814 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d72e9a-885f-4ad8-a1af-2e77e9f7e65b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:29.604091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d72e9a-885f-4ad8-a1af-2e77e9f7e65b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:29.604812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d72e9a-885f-4ad8-a1af-2e77e9f7e65b None admin] RBAC: Adding query filter params (role.id=290368e3589e425e99a9f4dbfe2b3823, scope.project.id=f0388fccf3c54977b8da1e55ff5bdc24, user.id=5acfe6e3e7be4573aab3cde32be1d7ea) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:29.605126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d72e9a-885f-4ad8-a1af-2e77e9f7e65b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:29.606664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d72e9a-885f-4ad8-a1af-2e77e9f7e65b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:29.612746 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 63/129] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:34:29 2022] GET /identity/v3/role_assignments?user.id=5acfe6e3e7be4573aab3cde32be1d7ea&scope.project.id=f0388fccf3c54977b8da1e55ff5bdc24&role.id=290368e3589e425e99a9f4dbfe2b3823 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:30.187672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908893e-04cd-4e1c-9c66-5a2655eb258d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:30.187933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908893e-04cd-4e1c-9c66-5a2655eb258d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:30.188230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908893e-04cd-4e1c-9c66-5a2655eb258d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:30.189561 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 67/130] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:30.194911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99258a8e-2cc7-41fb-a772-9dc0c919e457 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:30.195205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99258a8e-2cc7-41fb-a772-9dc0c919e457 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:30.195479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99258a8e-2cc7-41fb-a772-9dc0c919e457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:30.214029 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-99258a8e-2cc7-41fb-a772-9dc0c919e457 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:30.215262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-99258a8e-2cc7-41fb-a772-9dc0c919e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:30.286443 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 64/131] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:30.470348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e902315-7ca5-4814-a4ac-5008db42eb2c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:30.470899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e902315-7ca5-4814-a4ac-5008db42eb2c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:30.470899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e902315-7ca5-4814-a4ac-5008db42eb2c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:30.472200 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 68/132] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:30.477512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943901e-a09b-4ff1-8a6e-20b1239be70d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:30.477940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943901e-a09b-4ff1-8a6e-20b1239be70d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:30.478062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943901e-a09b-4ff1-8a6e-20b1239be70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:30.484874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:30.498151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f943901e-a09b-4ff1-8a6e-20b1239be70d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:30.499357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f943901e-a09b-4ff1-8a6e-20b1239be70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:30.553727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e902315-7ca5-4814-a4ac-5008db42eb2c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:30.566603 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 65/133] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:30.660297 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e902315-7ca5-4814-a4ac-5008db42eb2c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:30.694656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e49b40-3611-4282-95f3-18a89dedd700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:30.695375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e49b40-3611-4282-95f3-18a89dedd700 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:30.695661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e49b40-3611-4282-95f3-18a89dedd700 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:30.695935 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e49b40-3611-4282-95f3-18a89dedd700 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:30.697153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e49b40-3611-4282-95f3-18a89dedd700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:30.698667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e49b40-3611-4282-95f3-18a89dedd700 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:30.700272 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 69/134] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:30.705993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943901e-a09b-4ff1-8a6e-20b1239be70d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:30.802828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943901e-a09b-4ff1-8a6e-20b1239be70d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:30.827064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e49b40-3611-4282-95f3-18a89dedd700 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:30.833798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:30.840059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176b01d-4f95-4607-84e8-368ebc9daa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:30.840709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176b01d-4f95-4607-84e8-368ebc9daa19 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:30.840988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176b01d-4f95-4607-84e8-368ebc9daa19 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:30.841257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176b01d-4f95-4607-84e8-368ebc9daa19 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:30.841912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176b01d-4f95-4607-84e8-368ebc9daa19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:30.843403 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176b01d-4f95-4607-84e8-368ebc9daa19 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:30.847328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:30.869591 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 66/135] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:34:30 2022] POST /identity/v3/users => generated 283 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:30.936410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e49b40-3611-4282-95f3-18a89dedd700 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:30.975576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cea2c-5138-4365-860b-1f0843f9ef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:30.976222 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cea2c-5138-4365-860b-1f0843f9ef26 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:30.976500 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cea2c-5138-4365-860b-1f0843f9ef26 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:30.976775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cea2c-5138-4365-860b-1f0843f9ef26 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:30.988449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cea2c-5138-4365-860b-1f0843f9ef26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1b2cea2c-5138-4365-860b-1f0843f9ef26 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:30.993804 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:30.994917 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:30.995693 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 70/136] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:34:31.001322 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176b01d-4f95-4607-84e8-368ebc9daa19 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:31.109051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176b01d-4f95-4607-84e8-368ebc9daa19 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:31.146919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46349391-6b64-47f9-904c-c7bb44b698fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:31.147551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46349391-6b64-47f9-904c-c7bb44b698fa None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:31.147832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46349391-6b64-47f9-904c-c7bb44b698fa None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:31.148206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46349391-6b64-47f9-904c-c7bb44b698fa None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:31.148887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46349391-6b64-47f9-904c-c7bb44b698fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:31.149200 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46349391-6b64-47f9-904c-c7bb44b698fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:31.150616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46349391-6b64-47f9-904c-c7bb44b698fa None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:31.159719 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 67/137] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:31.165687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cea2c-5138-4365-860b-1f0843f9ef26 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:31.250661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:31.279778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cea2c-5138-4365-860b-1f0843f9ef26 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:31.317259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5534a-e6a5-4f9d-b696-6e70d62430fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:31.317853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5534a-e6a5-4f9d-b696-6e70d62430fa None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:31.318135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5534a-e6a5-4f9d-b696-6e70d62430fa None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:31.318411 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5534a-e6a5-4f9d-b696-6e70d62430fa None admin] PATH_INFO: `/v3/users/a59abf99e567496e82501c2b0cc22f6f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:31.319610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5534a-e6a5-4f9d-b696-6e70d62430fa None admin] RBAC: Authorizing `identity:get_user(user_id=a59abf99e567496e82501c2b0cc22f6f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:31.321076 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5534a-e6a5-4f9d-b696-6e70d62430fa None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:31.323106 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 71/138] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:31 2022] GET /identity/v3/users/a59abf99e567496e82501c2b0cc22f6f => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:31.328864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46349391-6b64-47f9-904c-c7bb44b698fa None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:31.389260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:31.445030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46349391-6b64-47f9-904c-c7bb44b698fa None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:31.481422 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38716c-ab55-4dcb-b598-6ec64e82e661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:31.482098 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38716c-ab55-4dcb-b598-6ec64e82e661 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:31.482384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38716c-ab55-4dcb-b598-6ec64e82e661 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:31.482663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38716c-ab55-4dcb-b598-6ec64e82e661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:31.559532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38716c-ab55-4dcb-b598-6ec64e82e661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:31.561060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38716c-ab55-4dcb-b598-6ec64e82e661 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:31.610776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:31.662777 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 68/139] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:31.668402 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5534a-e6a5-4f9d-b696-6e70d62430fa None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:31.773302 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5534a-e6a5-4f9d-b696-6e70d62430fa None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:31.808408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8a75d-3460-4826-a404-fc44aadec999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:31.809476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8a75d-3460-4826-a404-fc44aadec999 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:31.810018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8a75d-3460-4826-a404-fc44aadec999 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:31.810917 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8a75d-3460-4826-a404-fc44aadec999 None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:31.812310 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8a75d-3460-4826-a404-fc44aadec999 None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:31.813967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8a75d-3460-4826-a404-fc44aadec999 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:31.815984 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 72/140] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:31 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:31.822148 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38716c-ab55-4dcb-b598-6ec64e82e661 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:31.932656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38716c-ab55-4dcb-b598-6ec64e82e661 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:31.968600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f036120-83ed-4f9c-bbb8-81f0a945f3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:31.969578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f036120-83ed-4f9c-bbb8-81f0a945f3a7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:31.970059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f036120-83ed-4f9c-bbb8-81f0a945f3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:31.970535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f036120-83ed-4f9c-bbb8-81f0a945f3a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:31.971511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f036120-83ed-4f9c-bbb8-81f0a945f3a7 None admin] RBAC: Adding query filter params (role.id=b96f3faefe1c453fb20deea4530c43f0, scope.project.id=538dc9f6865b4b1a932b75e3be851bb7, user.id=a59abf99e567496e82501c2b0cc22f6f) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:31.972015 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f036120-83ed-4f9c-bbb8-81f0a945f3a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:31.973826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f036120-83ed-4f9c-bbb8-81f0a945f3a7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:31.980622 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 69/141] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:34:31 2022] GET /identity/v3/role_assignments?user.id=a59abf99e567496e82501c2b0cc22f6f&scope.project.id=538dc9f6865b4b1a932b75e3be851bb7&role.id=b96f3faefe1c453fb20deea4530c43f0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:33.127737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46bde4-6bc3-4bc9-8f2c-8ed9e8838510 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.128293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46bde4-6bc3-4bc9-8f2c-8ed9e8838510 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.128293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46bde4-6bc3-4bc9-8f2c-8ed9e8838510 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.129659 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 73/142] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:33.134995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8c84e-2b7b-464d-8986-e50c692b5b8a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.135437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8c84e-2b7b-464d-8986-e50c692b5b8a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.135515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8c84e-2b7b-464d-8986-e50c692b5b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.154227 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2df8c84e-2b7b-464d-8986-e50c692b5b8a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:33.155456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2df8c84e-2b7b-464d-8986-e50c692b5b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:33.167360 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:33.232382 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 70/143] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:33.286475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00d06e-4fc0-4710-9f4a-927951aae159 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.287160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00d06e-4fc0-4710-9f4a-927951aae159 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.287440 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00d06e-4fc0-4710-9f4a-927951aae159 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.288866 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 74/144] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:33.294426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d9ef9-76a8-40b8-bd64-b2acadc1cc82 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.294677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d9ef9-76a8-40b8-bd64-b2acadc1cc82 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.295001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d9ef9-76a8-40b8-bd64-b2acadc1cc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.314975 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-343d9ef9-76a8-40b8-bd64-b2acadc1cc82 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:33.316193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-343d9ef9-76a8-40b8-bd64-b2acadc1cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:33.387937 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 71/145] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:33.429742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cf5b0-7b69-4890-8be4-4282032c7cb2 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.430115 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cf5b0-7b69-4890-8be4-4282032c7cb2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.430470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cf5b0-7b69-4890-8be4-4282032c7cb2 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.434828 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 75/146] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:33.437539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed706eaf-1ffa-4793-8372-9ae57ef984f9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.437829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed706eaf-1ffa-4793-8372-9ae57ef984f9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.438111 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed706eaf-1ffa-4793-8372-9ae57ef984f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.456729 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ed706eaf-1ffa-4793-8372-9ae57ef984f9 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:33.457899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ed706eaf-1ffa-4793-8372-9ae57ef984f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:33.498441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cf5b0-7b69-4890-8be4-4282032c7cb2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:33.516133 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:33.528694 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 72/147] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:33.605372 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cf5b0-7b69-4890-8be4-4282032c7cb2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:33.638752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9f28e-0c00-4467-8821-b0b86a1250aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:33.639309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9f28e-0c00-4467-8821-b0b86a1250aa None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.639591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9f28e-0c00-4467-8821-b0b86a1250aa None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.639868 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9f28e-0c00-4467-8821-b0b86a1250aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.650087 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9f28e-0c00-4467-8821-b0b86a1250aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:33.653822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed706eaf-1ffa-4793-8372-9ae57ef984f9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-90b9f28e-0c00-4467-8821-b0b86a1250aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:33.655463 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:33.656381 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:33.659838 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 76/148] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:34:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:33.662392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:33.662505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9f28e-0c00-4467-8821-b0b86a1250aa None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:33.739246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:33.763487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed706eaf-1ffa-4793-8372-9ae57ef984f9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:33.766266 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9f28e-0c00-4467-8821-b0b86a1250aa None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:33.799502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af689e-7345-494e-b813-0faa35fd2b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:33.800142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af689e-7345-494e-b813-0faa35fd2b45 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.800417 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af689e-7345-494e-b813-0faa35fd2b45 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.800686 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af689e-7345-494e-b813-0faa35fd2b45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.801190 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f291b2c-963e-464f-834d-37b84b50fbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:33.801803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f291b2c-963e-464f-834d-37b84b50fbbc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.802082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f291b2c-963e-464f-834d-37b84b50fbbc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.802358 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f291b2c-963e-464f-834d-37b84b50fbbc None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.803054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f291b2c-963e-464f-834d-37b84b50fbbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:33.803363 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f291b2c-963e-464f-834d-37b84b50fbbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:33.804749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f291b2c-963e-464f-834d-37b84b50fbbc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:33.811128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af689e-7345-494e-b813-0faa35fd2b45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:33.813056 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 77/149] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:33.813825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f291b2c-963e-464f-834d-37b84b50fbbc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e6af689e-7345-494e-b813-0faa35fd2b45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:33.816070 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:33.816952 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:33.817737 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 73/150] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:34:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:34:33.818627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af689e-7345-494e-b813-0faa35fd2b45 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:33.877165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:33.913428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af689e-7345-494e-b813-0faa35fd2b45 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:33.915101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f291b2c-963e-464f-834d-37b84b50fbbc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:33.946535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8c8d4-7ef6-4e0b-b8bc-553edeec0a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:33.947193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8c8d4-7ef6-4e0b-b8bc-553edeec0a94 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.947476 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8c8d4-7ef6-4e0b-b8bc-553edeec0a94 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.947751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8c8d4-7ef6-4e0b-b8bc-553edeec0a94 None admin] PATH_INFO: `/v3/users/554573b99ac04ef1b3e8e352ab9a5601` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.949546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf705e-6b6b-44da-b672-52847d41f0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:33.950278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf705e-6b6b-44da-b672-52847d41f0f0 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:33.950568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf705e-6b6b-44da-b672-52847d41f0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:33.950870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf705e-6b6b-44da-b672-52847d41f0f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:33.951975 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf705e-6b6b-44da-b672-52847d41f0f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:33.953421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf705e-6b6b-44da-b672-52847d41f0f0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:33.955059 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 78/151] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:33.955915 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf705e-6b6b-44da-b672-52847d41f0f0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:33.959981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:33.961627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8c8d4-7ef6-4e0b-b8bc-553edeec0a94 None admin] RBAC: Authorizing `identity:get_user(user_id=554573b99ac04ef1b3e8e352ab9a5601)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:33.963364 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8c8d4-7ef6-4e0b-b8bc-553edeec0a94 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:33.969550 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 74/152] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:33 2022] GET /identity/v3/users/554573b99ac04ef1b3e8e352ab9a5601 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:33.969550 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8c8d4-7ef6-4e0b-b8bc-553edeec0a94 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:34.058944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf705e-6b6b-44da-b672-52847d41f0f0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:34.065740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8c8d4-7ef6-4e0b-b8bc-553edeec0a94 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:34.087450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:34.091898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61749155-07d7-4dce-9857-f43596e9801f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:34.092529 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61749155-07d7-4dce-9857-f43596e9801f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:34.092809 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61749155-07d7-4dce-9857-f43596e9801f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:34.093082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61749155-07d7-4dce-9857-f43596e9801f None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:34.093745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61749155-07d7-4dce-9857-f43596e9801f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:34.094053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61749155-07d7-4dce-9857-f43596e9801f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:34.095471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61749155-07d7-4dce-9857-f43596e9801f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:34.099403 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd10c9-a7d4-4d26-86ca-def9e9dc9a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:34.100006 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd10c9-a7d4-4d26-86ca-def9e9dc9a59 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:34.100553 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd10c9-a7d4-4d26-86ca-def9e9dc9a59 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:34.100829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd10c9-a7d4-4d26-86ca-def9e9dc9a59 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:34.101470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd10c9-a7d4-4d26-86ca-def9e9dc9a59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:34.102927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd10c9-a7d4-4d26-86ca-def9e9dc9a59 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:34.103371 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 79/153] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:34:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:34.107156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61749155-07d7-4dce-9857-f43596e9801f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:34.124822 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 75/154] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:34:33 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:34.125625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd10c9-a7d4-4d26-86ca-def9e9dc9a59 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:34.182182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:34.225496 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61749155-07d7-4dce-9857-f43596e9801f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:34.247479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd10c9-a7d4-4d26-86ca-def9e9dc9a59 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:34.261592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41863e8a-3319-40d0-a591-3a6a49a1f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:34.262248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41863e8a-3319-40d0-a591-3a6a49a1f183 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:34.262548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41863e8a-3319-40d0-a591-3a6a49a1f183 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:34.262891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41863e8a-3319-40d0-a591-3a6a49a1f183 None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:34.264036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41863e8a-3319-40d0-a591-3a6a49a1f183 None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:34.267511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41863e8a-3319-40d0-a591-3a6a49a1f183 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:34.267511 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 80/155] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:34 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:34.273333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41863e8a-3319-40d0-a591-3a6a49a1f183 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:34.293698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23979a45-9d67-4212-bd48-948a46b95ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:34.294392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23979a45-9d67-4212-bd48-948a46b95ddd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:34.294669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23979a45-9d67-4212-bd48-948a46b95ddd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:34.294967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23979a45-9d67-4212-bd48-948a46b95ddd None admin] PATH_INFO: `/v3/users/554573b99ac04ef1b3e8e352ab9a5601` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:34.296103 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23979a45-9d67-4212-bd48-948a46b95ddd None admin] RBAC: Authorizing `identity:get_user(user_id=554573b99ac04ef1b3e8e352ab9a5601)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:34.297584 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23979a45-9d67-4212-bd48-948a46b95ddd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:34.299413 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 76/156] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:34 2022] GET /identity/v3/users/554573b99ac04ef1b3e8e352ab9a5601 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:34.305246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23979a45-9d67-4212-bd48-948a46b95ddd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:34.355569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:34.387750 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41863e8a-3319-40d0-a591-3a6a49a1f183 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:34.417993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23979a45-9d67-4212-bd48-948a46b95ddd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:34.432734 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39668dad-27d6-4bfd-ac8a-3bf4ff06346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:34.433701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39668dad-27d6-4bfd-ac8a-3bf4ff06346f None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:34.433987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39668dad-27d6-4bfd-ac8a-3bf4ff06346f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:34.434291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39668dad-27d6-4bfd-ac8a-3bf4ff06346f None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62/users/554573b99ac04ef1b3e8e352ab9a5601/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:34.436956 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39668dad-27d6-4bfd-ac8a-3bf4ff06346f None admin] RBAC: Authorizing `identity:create_grant(project_id=157d82dd257b4e0b929f245d80d41b62, user_id=554573b99ac04ef1b3e8e352ab9a5601, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:34.438414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39668dad-27d6-4bfd-ac8a-3bf4ff06346f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:34.442344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:34.459426 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 81/157] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:34:34 2022] PUT /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62/users/554573b99ac04ef1b3e8e352ab9a5601/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:34:34.465778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e948d4-89d6-4387-a147-fefa39bb032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:34.466604 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e948d4-89d6-4387-a147-fefa39bb032e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:34.467070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e948d4-89d6-4387-a147-fefa39bb032e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:34.467492 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e948d4-89d6-4387-a147-fefa39bb032e None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:34.469507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e948d4-89d6-4387-a147-fefa39bb032e None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:34.471037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e948d4-89d6-4387-a147-fefa39bb032e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:34.472902 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 77/158] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:34 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:34.478842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39668dad-27d6-4bfd-ac8a-3bf4ff06346f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:34.592128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39668dad-27d6-4bfd-ac8a-3bf4ff06346f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:34.629679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394f567-e9fa-4fbe-a8e8-47742d584066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:34.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394f567-e9fa-4fbe-a8e8-47742d584066 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:34.631182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394f567-e9fa-4fbe-a8e8-47742d584066 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:34.631625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394f567-e9fa-4fbe-a8e8-47742d584066 None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62/users/554573b99ac04ef1b3e8e352ab9a5601/roles/5cd206a783e44ac2829c7661b34bb6a7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:34.634062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394f567-e9fa-4fbe-a8e8-47742d584066 None admin] RBAC: Authorizing `identity:create_grant(project_id=157d82dd257b4e0b929f245d80d41b62, user_id=554573b99ac04ef1b3e8e352ab9a5601, role_id=5cd206a783e44ac2829c7661b34bb6a7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:34.635954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394f567-e9fa-4fbe-a8e8-47742d584066 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:34.652967 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 82/159] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:34:34 2022] PUT /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62/users/554573b99ac04ef1b3e8e352ab9a5601/roles/5cd206a783e44ac2829c7661b34bb6a7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:34:36.410571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef79274-e3cf-4f78-9dbb-092321e450be None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:36.411165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef79274-e3cf-4f78-9dbb-092321e450be None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:36.411165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef79274-e3cf-4f78-9dbb-092321e450be None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:36.413081 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 78/160] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:36.418210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8093ba-091c-49ee-afa4-261cd9b9b021 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:36.418619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8093ba-091c-49ee-afa4-261cd9b9b021 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:36.418895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8093ba-091c-49ee-afa4-261cd9b9b021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:36.438036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0e8093ba-091c-49ee-afa4-261cd9b9b021 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:36.439381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0e8093ba-091c-49ee-afa4-261cd9b9b021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:36.498134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:36.517221 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 83/161] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:36.568688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34901b-743d-4687-83be-b3116be3d56a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:36.568878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34901b-743d-4687-83be-b3116be3d56a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:36.569152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34901b-743d-4687-83be-b3116be3d56a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:36.570692 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 79/162] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:36.575988 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45afd95-ef98-4a46-a440-9c2856b8d74f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:36.576278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45afd95-ef98-4a46-a440-9c2856b8d74f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:36.576552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45afd95-ef98-4a46-a440-9c2856b8d74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:36.594985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c45afd95-ef98-4a46-a440-9c2856b8d74f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:36.596173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c45afd95-ef98-4a46-a440-9c2856b8d74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:36.652934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690419f9-3c66-47b5-b4eb-2f95aaab8181 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:36.653240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690419f9-3c66-47b5-b4eb-2f95aaab8181 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:36.653514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690419f9-3c66-47b5-b4eb-2f95aaab8181 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:36.655182 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 80/163] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:36.666286 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 84/164] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:36.670522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b36ce-9d99-4472-a11a-f3a64c1d63d2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:36.672036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b36ce-9d99-4472-a11a-f3a64c1d63d2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:36.672036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b36ce-9d99-4472-a11a-f3a64c1d63d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:36.690254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-361b36ce-9d99-4472-a11a-f3a64c1d63d2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:36.691478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-361b36ce-9d99-4472-a11a-f3a64c1d63d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:36.764269 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 81/165] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:36.792726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45afd95-ef98-4a46-a440-9c2856b8d74f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:36.832173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c83c1-8cc5-4c92-a17a-8f9849fca94d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:36.832362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c83c1-8cc5-4c92-a17a-8f9849fca94d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:36.832643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c83c1-8cc5-4c92-a17a-8f9849fca94d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:36.834133 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 82/166] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:34:36.839277 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc9d24-9a27-42f1-9525-9359eeec819f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:36.839660 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc9d24-9a27-42f1-9525-9359eeec819f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:36.840020 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc9d24-9a27-42f1-9525-9359eeec819f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:36.861429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5dbc9d24-9a27-42f1-9525-9359eeec819f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:36.862914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5dbc9d24-9a27-42f1-9525-9359eeec819f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:36.897945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45afd95-ef98-4a46-a440-9c2856b8d74f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:36.912585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:36.917624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:36.944516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ea22e-0c4b-4a2e-8bc5-8c3a0ee400a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:36.950803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ea22e-0c4b-4a2e-8bc5-8c3a0ee400a3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:36.950803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ea22e-0c4b-4a2e-8bc5-8c3a0ee400a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:36.950803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ea22e-0c4b-4a2e-8bc5-8c3a0ee400a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:36.951187 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 83/167] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:36.955297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc9d24-9a27-42f1-9525-9359eeec819f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:36.957046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ea22e-0c4b-4a2e-8bc5-8c3a0ee400a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9b4ea22e-0c4b-4a2e-8bc5-8c3a0ee400a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:36.962889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:36.963920 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:36.964746 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 85/168] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:36.970074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ea22e-0c4b-4a2e-8bc5-8c3a0ee400a3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.074304 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc9d24-9a27-42f1-9525-9359eeec819f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.082296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ea22e-0c4b-4a2e-8bc5-8c3a0ee400a3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.108749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d098-dab2-4f90-aa5c-1c8c65d96145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.109391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d098-dab2-4f90-aa5c-1c8c65d96145 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.109677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d098-dab2-4f90-aa5c-1c8c65d96145 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.109944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d098-dab2-4f90-aa5c-1c8c65d96145 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.111076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d098-dab2-4f90-aa5c-1c8c65d96145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:37.112539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58d098-dab2-4f90-aa5c-1c8c65d96145 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:37.114310 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 84/169] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:37.115832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d098-dab2-4f90-aa5c-1c8c65d96145 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.117886 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f2a74-878e-4307-9fbd-efd4626db6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.118694 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f2a74-878e-4307-9fbd-efd4626db6a4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.118849 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f2a74-878e-4307-9fbd-efd4626db6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.119121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f2a74-878e-4307-9fbd-efd4626db6a4 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.119782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f2a74-878e-4307-9fbd-efd4626db6a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:37.120079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f2a74-878e-4307-9fbd-efd4626db6a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:37.121457 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f2a74-878e-4307-9fbd-efd4626db6a4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:37.129828 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 86/170] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:37.130578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f2a74-878e-4307-9fbd-efd4626db6a4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.167568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:37.172245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:37.229365 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d098-dab2-4f90-aa5c-1c8c65d96145 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.233915 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f2a74-878e-4307-9fbd-efd4626db6a4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.264231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376d81e-5378-4555-a6ae-6e59c25bde8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.264874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376d81e-5378-4555-a6ae-6e59c25bde8c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.265153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376d81e-5378-4555-a6ae-6e59c25bde8c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.265427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376d81e-5378-4555-a6ae-6e59c25bde8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.269008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86996e86-6184-4a34-8bc5-0ec67cc28d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.269674 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86996e86-6184-4a34-8bc5-0ec67cc28d7a None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.270104 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86996e86-6184-4a34-8bc5-0ec67cc28d7a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.270507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86996e86-6184-4a34-8bc5-0ec67cc28d7a None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.271187 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86996e86-6184-4a34-8bc5-0ec67cc28d7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:37.272658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86996e86-6184-4a34-8bc5-0ec67cc28d7a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:37.275504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376d81e-5378-4555-a6ae-6e59c25bde8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-d376d81e-5378-4555-a6ae-6e59c25bde8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:37.280401 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:37.281282 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:37.282097 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 85/171] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:34:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:37.282776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376d81e-5378-4555-a6ae-6e59c25bde8c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.297944 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 87/172] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:34:37 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:37.298975 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86996e86-6184-4a34-8bc5-0ec67cc28d7a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.401527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:37.408692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376d81e-5378-4555-a6ae-6e59c25bde8c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.410478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:37.413897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86996e86-6184-4a34-8bc5-0ec67cc28d7a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.450668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffea9a5-3300-46fe-8a89-2afff59eb8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.451315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a460fb-1118-4371-a41c-fd9f62ab9fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.451668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffea9a5-3300-46fe-8a89-2afff59eb8fc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.451668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffea9a5-3300-46fe-8a89-2afff59eb8fc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.451990 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a460fb-1118-4371-a41c-fd9f62ab9fae None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.452113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffea9a5-3300-46fe-8a89-2afff59eb8fc None admin] PATH_INFO: `/v3/users/30e4cf1f4c0e4c7695f5bbce0e023e8d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.452187 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a460fb-1118-4371-a41c-fd9f62ab9fae None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.452421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a460fb-1118-4371-a41c-fd9f62ab9fae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.464928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a460fb-1118-4371-a41c-fd9f62ab9fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:37.467470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffea9a5-3300-46fe-8a89-2afff59eb8fc None admin] RBAC: Authorizing `identity:get_user(user_id=30e4cf1f4c0e4c7695f5bbce0e023e8d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:37.469000 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffea9a5-3300-46fe-8a89-2afff59eb8fc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e3a460fb-1118-4371-a41c-fd9f62ab9fae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:37.470411 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:37.476092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:37.477069 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:37.477690 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 86/173] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:37 2022] GET /identity/v3/users/30e4cf1f4c0e4c7695f5bbce0e023e8d => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:37.477768 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 88/174] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:34:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:37.477834 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffea9a5-3300-46fe-8a89-2afff59eb8fc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.477949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a460fb-1118-4371-a41c-fd9f62ab9fae None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.598446 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffea9a5-3300-46fe-8a89-2afff59eb8fc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.605382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a460fb-1118-4371-a41c-fd9f62ab9fae None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.641099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed962c4c-100e-49a9-a23e-150f51250098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.642024 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed962c4c-100e-49a9-a23e-150f51250098 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.642445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed962c4c-100e-49a9-a23e-150f51250098 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.642759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed962c4c-100e-49a9-a23e-150f51250098 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.643409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed962c4c-100e-49a9-a23e-150f51250098 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:37.643708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed962c4c-100e-49a9-a23e-150f51250098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:37.645094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed962c4c-100e-49a9-a23e-150f51250098 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:37.646560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc2b0-664e-494a-af53-65a4510ebf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.647561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc2b0-664e-494a-af53-65a4510ebf3a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.647852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc2b0-664e-494a-af53-65a4510ebf3a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.648126 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc2b0-664e-494a-af53-65a4510ebf3a None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.649250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc2b0-664e-494a-af53-65a4510ebf3a None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:37.651735 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc2b0-664e-494a-af53-65a4510ebf3a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:37.653814 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 89/175] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:37 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:37.658474 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 87/176] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:37.658597 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc2b0-664e-494a-af53-65a4510ebf3a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.659566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed962c4c-100e-49a9-a23e-150f51250098 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.685320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:37.686753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:37.771875 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed962c4c-100e-49a9-a23e-150f51250098 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.773293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc2b0-664e-494a-af53-65a4510ebf3a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.812275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e48d90-37d8-4c64-b2f3-5d353be4b20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.813145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e48d90-37d8-4c64-b2f3-5d353be4b20c None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.813529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e48d90-37d8-4c64-b2f3-5d353be4b20c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.813806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e48d90-37d8-4c64-b2f3-5d353be4b20c None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62/users/30e4cf1f4c0e4c7695f5bbce0e023e8d/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.815821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986d282-7a44-4702-9f2a-2878a2a837c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.816282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e48d90-37d8-4c64-b2f3-5d353be4b20c None admin] RBAC: Authorizing `identity:create_grant(project_id=157d82dd257b4e0b929f245d80d41b62, user_id=30e4cf1f4c0e4c7695f5bbce0e023e8d, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:37.816549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986d282-7a44-4702-9f2a-2878a2a837c5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.816891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986d282-7a44-4702-9f2a-2878a2a837c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.817216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986d282-7a44-4702-9f2a-2878a2a837c5 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:37.817787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e48d90-37d8-4c64-b2f3-5d353be4b20c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:37.817996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986d282-7a44-4702-9f2a-2878a2a837c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:37.818359 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986d282-7a44-4702-9f2a-2878a2a837c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:37.819833 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986d282-7a44-4702-9f2a-2878a2a837c5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:37.830569 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 90/177] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:34:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:37.831509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986d282-7a44-4702-9f2a-2878a2a837c5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.834145 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 88/178] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:34:37 2022] PUT /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62/users/30e4cf1f4c0e4c7695f5bbce0e023e8d/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:37.836350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e48d90-37d8-4c64-b2f3-5d353be4b20c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:37.926494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:37.947875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986d282-7a44-4702-9f2a-2878a2a837c5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:37.992411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:37.995051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095afa4a-0882-4d6b-8a95-4c8439e62464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:37.998148 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095afa4a-0882-4d6b-8a95-4c8439e62464 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:37.998470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095afa4a-0882-4d6b-8a95-4c8439e62464 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:37.998764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095afa4a-0882-4d6b-8a95-4c8439e62464 None admin] PATH_INFO: `/v3/users/554573b99ac04ef1b3e8e352ab9a5601` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:38.000122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095afa4a-0882-4d6b-8a95-4c8439e62464 None admin] RBAC: Authorizing `identity:get_user(user_id=554573b99ac04ef1b3e8e352ab9a5601)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:38.002240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095afa4a-0882-4d6b-8a95-4c8439e62464 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:38.003981 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 91/179] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:37 2022] GET /identity/v3/users/554573b99ac04ef1b3e8e352ab9a5601 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:38.019813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095afa4a-0882-4d6b-8a95-4c8439e62464 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:38.024134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e48d90-37d8-4c64-b2f3-5d353be4b20c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:38.064261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe86f9-a84f-4a19-ac5c-c4d48fcacf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:38.065066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe86f9-a84f-4a19-ac5c-c4d48fcacf8f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:38.065431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe86f9-a84f-4a19-ac5c-c4d48fcacf8f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:38.065799 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe86f9-a84f-4a19-ac5c-c4d48fcacf8f None admin] PATH_INFO: `/v3/users/554573b99ac04ef1b3e8e352ab9a5601` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:38.067574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe86f9-a84f-4a19-ac5c-c4d48fcacf8f None admin] RBAC: Authorizing `identity:get_user(user_id=554573b99ac04ef1b3e8e352ab9a5601)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:38.068796 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe86f9-a84f-4a19-ac5c-c4d48fcacf8f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:38.070882 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 89/180] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:37 2022] GET /identity/v3/users/554573b99ac04ef1b3e8e352ab9a5601 => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:38.076090 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe86f9-a84f-4a19-ac5c-c4d48fcacf8f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:38.152171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095afa4a-0882-4d6b-8a95-4c8439e62464 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:38.189344 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe86f9-a84f-4a19-ac5c-c4d48fcacf8f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:38.220604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01875622-8672-4978-9ff5-64b6048b1440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:38.221231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01875622-8672-4978-9ff5-64b6048b1440 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:38.221508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01875622-8672-4978-9ff5-64b6048b1440 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:38.221766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01875622-8672-4978-9ff5-64b6048b1440 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:38.225197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9aec3-01e9-4e90-b46a-d3d746d52f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:38.226034 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9aec3-01e9-4e90-b46a-d3d746d52f18 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:38.226450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9aec3-01e9-4e90-b46a-d3d746d52f18 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:38.226911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9aec3-01e9-4e90-b46a-d3d746d52f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:38.246549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:38.265569 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:38.300959 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9aec3-01e9-4e90-b46a-d3d746d52f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:38.302784 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9aec3-01e9-4e90-b46a-d3d746d52f18 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:38.379806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01875622-8672-4978-9ff5-64b6048b1440 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:38.381292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01875622-8672-4978-9ff5-64b6048b1440 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:38.405171 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 90/181] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:38.414969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9aec3-01e9-4e90-b46a-d3d746d52f18 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:38.469170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:38.523962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9aec3-01e9-4e90-b46a-d3d746d52f18 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:38.561995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2044d-f769-42fc-ac8d-01c5a537c8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:38.562944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2044d-f769-42fc-ac8d-01c5a537c8ae None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:38.563362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2044d-f769-42fc-ac8d-01c5a537c8ae None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:38.563773 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2044d-f769-42fc-ac8d-01c5a537c8ae None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:38.565295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2044d-f769-42fc-ac8d-01c5a537c8ae None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:38.567069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2044d-f769-42fc-ac8d-01c5a537c8ae None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:38.569313 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 91/182] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:38 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:38.575269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2044d-f769-42fc-ac8d-01c5a537c8ae None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:38.595343 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 92/183] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:38.597556 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01875622-8672-4978-9ff5-64b6048b1440 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:38.683712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:38.733992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2044d-f769-42fc-ac8d-01c5a537c8ae None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:38.756842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:38.773906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01875622-8672-4978-9ff5-64b6048b1440 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:38.790300 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a42146-719b-42f0-b509-5d4bd509ca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:38.791208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42146-719b-42f0-b509-5d4bd509ca25 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:38.791626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42146-719b-42f0-b509-5d4bd509ca25 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:38.792036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a42146-719b-42f0-b509-5d4bd509ca25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:38.792921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42146-719b-42f0-b509-5d4bd509ca25 None admin] RBAC: Adding query filter params (role.id=5cd206a783e44ac2829c7661b34bb6a7, scope.project.id=157d82dd257b4e0b929f245d80d41b62, user.id=554573b99ac04ef1b3e8e352ab9a5601) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:38.793369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42146-719b-42f0-b509-5d4bd509ca25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:38.795715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a42146-719b-42f0-b509-5d4bd509ca25 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:38.805375 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 92/184] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:34:38 2022] GET /identity/v3/role_assignments?user.id=554573b99ac04ef1b3e8e352ab9a5601&scope.project.id=157d82dd257b4e0b929f245d80d41b62&role.id=5cd206a783e44ac2829c7661b34bb6a7 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:38.842469 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa545fd-c5f9-4303-8784-c1815b29f3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:38.843164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa545fd-c5f9-4303-8784-c1815b29f3cb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:38.843443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa545fd-c5f9-4303-8784-c1815b29f3cb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:38.843715 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa545fd-c5f9-4303-8784-c1815b29f3cb None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:38.847590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa545fd-c5f9-4303-8784-c1815b29f3cb None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:38.849092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa545fd-c5f9-4303-8784-c1815b29f3cb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:38.851149 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 93/185] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:38 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:38.857253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a42146-719b-42f0-b509-5d4bd509ca25 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:39.014881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a42146-719b-42f0-b509-5d4bd509ca25 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:39.060500 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb442fc-3bd9-445d-861b-888bcf84e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:39.061139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb442fc-3bd9-445d-861b-888bcf84e5db None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:39.061425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb442fc-3bd9-445d-861b-888bcf84e5db None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:39.061704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb442fc-3bd9-445d-861b-888bcf84e5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:39.062460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb442fc-3bd9-445d-861b-888bcf84e5db None admin] RBAC: Adding query filter params (role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.project.id=157d82dd257b4e0b929f245d80d41b62, user.id=554573b99ac04ef1b3e8e352ab9a5601) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:39.062792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb442fc-3bd9-445d-861b-888bcf84e5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:39.064299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb442fc-3bd9-445d-861b-888bcf84e5db None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:39.071077 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 93/186] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:34:38 2022] GET /identity/v3/role_assignments?user.id=554573b99ac04ef1b3e8e352ab9a5601&scope.project.id=157d82dd257b4e0b929f245d80d41b62&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:39.791859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2ea0b-2ea1-4cb3-b1d8-23466957ba04 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:39.792432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2ea0b-2ea1-4cb3-b1d8-23466957ba04 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:39.792432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2ea0b-2ea1-4cb3-b1d8-23466957ba04 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:39.794007 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 94/187] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:39.799259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99be11-0ca0-4ec8-8dec-af67a6ec441b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:39.799708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99be11-0ca0-4ec8-8dec-af67a6ec441b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:39.799784 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99be11-0ca0-4ec8-8dec-af67a6ec441b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:39.818021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8e99be11-0ca0-4ec8-8dec-af67a6ec441b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:39.819223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8e99be11-0ca0-4ec8-8dec-af67a6ec441b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:39.861913 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:39.890968 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 94/188] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:39.989953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915be72-f94f-4f02-82cc-98d57970a226 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:39.990272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915be72-f94f-4f02-82cc-98d57970a226 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:39.990539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915be72-f94f-4f02-82cc-98d57970a226 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:39.992070 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 95/189] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:39.997237 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18eca1-8d3d-4733-975f-dc155ea31555 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:39.997526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18eca1-8d3d-4733-975f-dc155ea31555 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:39.997805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18eca1-8d3d-4733-975f-dc155ea31555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.015831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1f18eca1-8d3d-4733-975f-dc155ea31555 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:40.017028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1f18eca1-8d3d-4733-975f-dc155ea31555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:40.084677 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 95/190] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:40.160424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915be72-f94f-4f02-82cc-98d57970a226 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:40.268360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915be72-f94f-4f02-82cc-98d57970a226 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:40.272950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0efc4b-e794-4f56-803c-ef9f89b90963 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.273240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0efc4b-e794-4f56-803c-ef9f89b90963 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.273515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0efc4b-e794-4f56-803c-ef9f89b90963 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.275039 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 96/191] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:40.280925 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e00c69-1beb-4fdf-86c2-316d516b374a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.281225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e00c69-1beb-4fdf-86c2-316d516b374a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.281458 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e00c69-1beb-4fdf-86c2-316d516b374a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.290813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:40.300807 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b8e00c69-1beb-4fdf-86c2-316d516b374a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:40.301955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b8e00c69-1beb-4fdf-86c2-316d516b374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:40.308885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2f1f1-70ab-49ee-b4d4-addecbaad6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:40.309712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2f1f1-70ab-49ee-b4d4-addecbaad6cb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.310003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2f1f1-70ab-49ee-b4d4-addecbaad6cb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.310265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2f1f1-70ab-49ee-b4d4-addecbaad6cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.320780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2f1f1-70ab-49ee-b4d4-addecbaad6cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c8f2f1f1-70ab-49ee-b4d4-addecbaad6cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:40.326068 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:40.327001 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:40.327797 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 96/192] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:40.333098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2f1f1-70ab-49ee-b4d4-addecbaad6cb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:40.372713 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 97/193] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:40.374308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e00c69-1beb-4fdf-86c2-316d516b374a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:40.411992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:40.435470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2f1f1-70ab-49ee-b4d4-addecbaad6cb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:40.471793 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82343d45-1378-4c9a-8ccc-ee7df9784942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:40.472391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82343d45-1378-4c9a-8ccc-ee7df9784942 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.472661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82343d45-1378-4c9a-8ccc-ee7df9784942 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.472922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82343d45-1378-4c9a-8ccc-ee7df9784942 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.473580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82343d45-1378-4c9a-8ccc-ee7df9784942 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:40.473877 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82343d45-1378-4c9a-8ccc-ee7df9784942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:40.475287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82343d45-1378-4c9a-8ccc-ee7df9784942 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:40.483633 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 97/194] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:40.484254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e00c69-1beb-4fdf-86c2-316d516b374a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:40.488949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82343d45-1378-4c9a-8ccc-ee7df9784942 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:40.523832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618de91c-c459-4b68-8724-874564977f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:40.524482 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618de91c-c459-4b68-8724-874564977f46 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.524760 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618de91c-c459-4b68-8724-874564977f46 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.525034 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618de91c-c459-4b68-8724-874564977f46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.526250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618de91c-c459-4b68-8724-874564977f46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:40.527794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618de91c-c459-4b68-8724-874564977f46 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:40.529561 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 98/195] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:40.534967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618de91c-c459-4b68-8724-874564977f46 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:40.536934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:40.599624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82343d45-1378-4c9a-8ccc-ee7df9784942 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:40.634145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618de91c-c459-4b68-8724-874564977f46 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:40.635771 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b80f23-dd60-45c2-bb58-fcde54eba128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:40.637071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b80f23-dd60-45c2-bb58-fcde54eba128 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.637071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b80f23-dd60-45c2-bb58-fcde54eba128 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.637071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b80f23-dd60-45c2-bb58-fcde54eba128 None admin] PATH_INFO: `/v3/users/aecf9e7820284aeebad7fefafb35d5d2` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.637695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:40.651092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b80f23-dd60-45c2-bb58-fcde54eba128 None admin] RBAC: Authorizing `identity:get_user(user_id=aecf9e7820284aeebad7fefafb35d5d2)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:40.652568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b80f23-dd60-45c2-bb58-fcde54eba128 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:40.654235 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 98/196] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:40 2022] GET /identity/v3/users/aecf9e7820284aeebad7fefafb35d5d2 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:40.655013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b80f23-dd60-45c2-bb58-fcde54eba128 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:40.672838 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097e68e-84db-4cd1-976b-b0b03fd0bdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:40.673456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097e68e-84db-4cd1-976b-b0b03fd0bdc3 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.673734 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097e68e-84db-4cd1-976b-b0b03fd0bdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.674042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097e68e-84db-4cd1-976b-b0b03fd0bdc3 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.674692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097e68e-84db-4cd1-976b-b0b03fd0bdc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:40.676189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097e68e-84db-4cd1-976b-b0b03fd0bdc3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:40.698097 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 99/197] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:34:40 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:40.698097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097e68e-84db-4cd1-976b-b0b03fd0bdc3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:40.762196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:40.767519 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b80f23-dd60-45c2-bb58-fcde54eba128 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:40.806884 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cfcd3-0909-4db9-b44f-4cfc98aeeff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:40.807606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cfcd3-0909-4db9-b44f-4cfc98aeeff5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.807799 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cfcd3-0909-4db9-b44f-4cfc98aeeff5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.808067 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02cfcd3-0909-4db9-b44f-4cfc98aeeff5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.814069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097e68e-84db-4cd1-976b-b0b03fd0bdc3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:40.820401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02cfcd3-0909-4db9-b44f-4cfc98aeeff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-b02cfcd3-0909-4db9-b44f-4cfc98aeeff5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:40.825765 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:40.826913 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:40.828293 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 99/198] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:40.832878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cfcd3-0909-4db9-b44f-4cfc98aeeff5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:40.854637 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38cf77-0389-47da-91a2-79b5a867f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:40.855331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38cf77-0389-47da-91a2-79b5a867f642 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.855607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38cf77-0389-47da-91a2-79b5a867f642 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.855877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38cf77-0389-47da-91a2-79b5a867f642 None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.857086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38cf77-0389-47da-91a2-79b5a867f642 None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:40.858566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38cf77-0389-47da-91a2-79b5a867f642 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:40.860301 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 100/199] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:40 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:40.866212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38cf77-0389-47da-91a2-79b5a867f642 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:40.902925 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:40.946160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02cfcd3-0909-4db9-b44f-4cfc98aeeff5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:40.991627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a427a3-ac9d-4322-8cb2-b87f7037c864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:40.991975 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38cf77-0389-47da-91a2-79b5a867f642 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:40.992247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a427a3-ac9d-4322-8cb2-b87f7037c864 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:40.992527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a427a3-ac9d-4322-8cb2-b87f7037c864 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:40.992798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a427a3-ac9d-4322-8cb2-b87f7037c864 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:40.993477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a427a3-ac9d-4322-8cb2-b87f7037c864 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:40.993778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a427a3-ac9d-4322-8cb2-b87f7037c864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:40.995297 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a427a3-ac9d-4322-8cb2-b87f7037c864 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:41.005745 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 100/200] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:41.012028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a427a3-ac9d-4322-8cb2-b87f7037c864 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:41.032207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d152f7c-8ca0-4432-8796-8b1638a40fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:41.032899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d152f7c-8ca0-4432-8796-8b1638a40fbd None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:41.033185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d152f7c-8ca0-4432-8796-8b1638a40fbd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:41.033464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d152f7c-8ca0-4432-8796-8b1638a40fbd None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62/users/aecf9e7820284aeebad7fefafb35d5d2/roles/b96f3faefe1c453fb20deea4530c43f0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:41.037988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d152f7c-8ca0-4432-8796-8b1638a40fbd None admin] RBAC: Authorizing `identity:create_grant(project_id=157d82dd257b4e0b929f245d80d41b62, user_id=aecf9e7820284aeebad7fefafb35d5d2, role_id=b96f3faefe1c453fb20deea4530c43f0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:41.040364 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d152f7c-8ca0-4432-8796-8b1638a40fbd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:41.044337 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:41.059771 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 101/201] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:34:40 2022] PUT /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62/users/aecf9e7820284aeebad7fefafb35d5d2/roles/b96f3faefe1c453fb20deea4530c43f0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:34:41.149276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a427a3-ac9d-4322-8cb2-b87f7037c864 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:41.188371 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9ec85-6dd0-4f43-8054-221f7eb217e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:41.189362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9ec85-6dd0-4f43-8054-221f7eb217e4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:41.189858 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9ec85-6dd0-4f43-8054-221f7eb217e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:41.190356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9ec85-6dd0-4f43-8054-221f7eb217e4 None admin] PATH_INFO: `/v3/users/30e4cf1f4c0e4c7695f5bbce0e023e8d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:41.191872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9ec85-6dd0-4f43-8054-221f7eb217e4 None admin] RBAC: Authorizing `identity:get_user(user_id=30e4cf1f4c0e4c7695f5bbce0e023e8d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:41.193586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc9ec85-6dd0-4f43-8054-221f7eb217e4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:41.195580 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 101/202] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:41 2022] GET /identity/v3/users/30e4cf1f4c0e4c7695f5bbce0e023e8d => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:41.201548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d152f7c-8ca0-4432-8796-8b1638a40fbd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:41.314625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:41.326471 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d152f7c-8ca0-4432-8796-8b1638a40fbd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:41.360927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96519d5-9aec-4aed-b589-30a180b46175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:41.361605 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96519d5-9aec-4aed-b589-30a180b46175 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:41.361890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96519d5-9aec-4aed-b589-30a180b46175 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:41.362169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96519d5-9aec-4aed-b589-30a180b46175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:41.432529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96519d5-9aec-4aed-b589-30a180b46175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:41.434013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96519d5-9aec-4aed-b589-30a180b46175 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:41.520668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:41.533273 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 102/203] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:41.539399 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9ec85-6dd0-4f43-8054-221f7eb217e4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:41.636843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:41.649917 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9ec85-6dd0-4f43-8054-221f7eb217e4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:41.684422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d351cb-b36b-48ed-b0d5-ba57468bcba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:41.685060 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d351cb-b36b-48ed-b0d5-ba57468bcba2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:41.685337 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d351cb-b36b-48ed-b0d5-ba57468bcba2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:41.685610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d351cb-b36b-48ed-b0d5-ba57468bcba2 None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:41.686755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d351cb-b36b-48ed-b0d5-ba57468bcba2 None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:41.688207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d351cb-b36b-48ed-b0d5-ba57468bcba2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:41.690022 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 102/204] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:41 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:41.695775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96519d5-9aec-4aed-b589-30a180b46175 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:41.799148 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96519d5-9aec-4aed-b589-30a180b46175 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:41.835923 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097d815-a72f-4862-8d43-b423825fb20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:41.836586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097d815-a72f-4862-8d43-b423825fb20d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:41.836816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097d815-a72f-4862-8d43-b423825fb20d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:41.837069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097d815-a72f-4862-8d43-b423825fb20d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:41.837811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097d815-a72f-4862-8d43-b423825fb20d None admin] RBAC: Adding query filter params (role.id=290368e3589e425e99a9f4dbfe2b3823, scope.project.id=157d82dd257b4e0b929f245d80d41b62, user.id=30e4cf1f4c0e4c7695f5bbce0e023e8d) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:41.838127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097d815-a72f-4862-8d43-b423825fb20d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:41.839621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097d815-a72f-4862-8d43-b423825fb20d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:41.845886 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 103/205] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:34:41 2022] GET /identity/v3/role_assignments?user.id=30e4cf1f4c0e4c7695f5bbce0e023e8d&scope.project.id=157d82dd257b4e0b929f245d80d41b62&role.id=290368e3589e425e99a9f4dbfe2b3823 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:43.012742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761aca7d-74f2-45be-929d-084228c65613 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.013274 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761aca7d-74f2-45be-929d-084228c65613 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.013576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761aca7d-74f2-45be-929d-084228c65613 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.014926 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 103/206] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:43.020225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d44c-b6f4-4e0d-b072-dc3aa330702a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.020648 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d44c-b6f4-4e0d-b072-dc3aa330702a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.020813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d44c-b6f4-4e0d-b072-dc3aa330702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.039649 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4e10d44c-b6f4-4e0d-b072-dc3aa330702a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:43.040810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4e10d44c-b6f4-4e0d-b072-dc3aa330702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:43.078710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524ccf5-bd36-4812-96b0-4f2fbbdbf244 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.079071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524ccf5-bd36-4812-96b0-4f2fbbdbf244 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.079285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524ccf5-bd36-4812-96b0-4f2fbbdbf244 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.080783 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 104/207] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:34:43.085894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c85bb4-24df-485e-8990-cf0abb4ef91c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.086178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c85bb4-24df-485e-8990-cf0abb4ef91c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.086432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c85bb4-24df-485e-8990-cf0abb4ef91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.090729 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:43.105031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-93c85bb4-24df-485e-8990-cf0abb4ef91c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:43.106256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-93c85bb4-24df-485e-8990-cf0abb4ef91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:43.111898 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 104/208] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:43.178914 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 105/209] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:43.268455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6de5fc-1b01-49b7-896e-3dae931b1857 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.268455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6de5fc-1b01-49b7-896e-3dae931b1857 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.268455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6de5fc-1b01-49b7-896e-3dae931b1857 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.268455 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 105/210] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:43.274314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c79adc-daaa-4e48-bc08-00de404a7521 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.274604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c79adc-daaa-4e48-bc08-00de404a7521 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.274905 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c79adc-daaa-4e48-bc08-00de404a7521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.296480 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f4c79adc-daaa-4e48-bc08-00de404a7521 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:43.297670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f4c79adc-daaa-4e48-bc08-00de404a7521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:43.363768 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 106/211] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:43.373450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6de5fc-1b01-49b7-896e-3dae931b1857 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:43.443218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c79adc-daaa-4e48-bc08-00de404a7521 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:43.465882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:43.478672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6de5fc-1b01-49b7-896e-3dae931b1857 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:43.513198 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed6858-a6fd-4e21-bbcb-c7e1349b58aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:43.513892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed6858-a6fd-4e21-bbcb-c7e1349b58aa None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.514172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed6858-a6fd-4e21-bbcb-c7e1349b58aa None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.514626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed6858-a6fd-4e21-bbcb-c7e1349b58aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.524686 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed6858-a6fd-4e21-bbcb-c7e1349b58aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-90ed6858-a6fd-4e21-bbcb-c7e1349b58aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:43.529599 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:43.530589 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:43.531435 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 106/212] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:43.536343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed6858-a6fd-4e21-bbcb-c7e1349b58aa None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:43.559193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c79adc-daaa-4e48-bc08-00de404a7521 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:43.580193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:43.594687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2489b4-bda0-4afe-840f-22e51bfef0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:43.595336 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2489b4-bda0-4afe-840f-22e51bfef0af None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.595714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2489b4-bda0-4afe-840f-22e51bfef0af None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.595982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2489b4-bda0-4afe-840f-22e51bfef0af None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.597504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2489b4-bda0-4afe-840f-22e51bfef0af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:43.599181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2489b4-bda0-4afe-840f-22e51bfef0af None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:43.600894 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 107/213] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:43.606320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2489b4-bda0-4afe-840f-22e51bfef0af None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:43.638709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed6858-a6fd-4e21-bbcb-c7e1349b58aa None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:43.672147 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840233d4-3316-4942-adfa-9c47859c7810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:43.672772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840233d4-3316-4942-adfa-9c47859c7810 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.673047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840233d4-3316-4942-adfa-9c47859c7810 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.673317 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840233d4-3316-4942-adfa-9c47859c7810 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.673981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840233d4-3316-4942-adfa-9c47859c7810 None admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:43.674283 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840233d4-3316-4942-adfa-9c47859c7810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:43.675713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840233d4-3316-4942-adfa-9c47859c7810 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:43.683522 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 107/214] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:43.684426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840233d4-3316-4942-adfa-9c47859c7810 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:43.689524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:43.706005 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2489b4-bda0-4afe-840f-22e51bfef0af None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:43.739158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f1160-2904-41a2-b1c4-3da1957cfc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:43.739758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f1160-2904-41a2-b1c4-3da1957cfc84 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.740046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f1160-2904-41a2-b1c4-3da1957cfc84 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.740311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f1160-2904-41a2-b1c4-3da1957cfc84 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.740957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f1160-2904-41a2-b1c4-3da1957cfc84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:43.742392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f1160-2904-41a2-b1c4-3da1957cfc84 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:43.763745 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 108/215] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:34:43 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:34:43.771847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f1160-2904-41a2-b1c4-3da1957cfc84 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:43.782907 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840233d4-3316-4942-adfa-9c47859c7810 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:43.786266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:43.829331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91855c73-e1de-4b6d-b2f9-8aada6f8fc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:43.830001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91855c73-e1de-4b6d-b2f9-8aada6f8fc7a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.830289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91855c73-e1de-4b6d-b2f9-8aada6f8fc7a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.830563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91855c73-e1de-4b6d-b2f9-8aada6f8fc7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.842591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91855c73-e1de-4b6d-b2f9-8aada6f8fc7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-91855c73-e1de-4b6d-b2f9-8aada6f8fc7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:43.848110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:43.849187 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:43.850097 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 108/216] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:43.855616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91855c73-e1de-4b6d-b2f9-8aada6f8fc7a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:43.898561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f1160-2904-41a2-b1c4-3da1957cfc84 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:43.935732 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4beb9-13d7-4cf3-9dbb-2e511f9114b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:43.936386 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4beb9-13d7-4cf3-9dbb-2e511f9114b3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:43.936657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4beb9-13d7-4cf3-9dbb-2e511f9114b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:43.936934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4beb9-13d7-4cf3-9dbb-2e511f9114b3 None admin] PATH_INFO: `/v3/users/6308242b9a9f49a387e447cab1244c22` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:43.948002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:43.952077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4beb9-13d7-4cf3-9dbb-2e511f9114b3 None admin] RBAC: Authorizing `identity:get_user(user_id=6308242b9a9f49a387e447cab1244c22)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:43.954271 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4beb9-13d7-4cf3-9dbb-2e511f9114b3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:43.959135 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 109/217] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:43 2022] GET /identity/v3/users/6308242b9a9f49a387e447cab1244c22 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:43.963825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4beb9-13d7-4cf3-9dbb-2e511f9114b3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:43.970770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91855c73-e1de-4b6d-b2f9-8aada6f8fc7a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:44.017340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de46e7-d3c8-447d-ba81-9094d29952ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:44.018325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de46e7-d3c8-447d-ba81-9094d29952ce None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:44.018607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de46e7-d3c8-447d-ba81-9094d29952ce None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:44.018908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de46e7-d3c8-447d-ba81-9094d29952ce None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:44.019573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de46e7-d3c8-447d-ba81-9094d29952ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:44.019874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de46e7-d3c8-447d-ba81-9094d29952ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:44.021290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de46e7-d3c8-447d-ba81-9094d29952ce None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:44.029587 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 109/218] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:44.036213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de46e7-d3c8-447d-ba81-9094d29952ce None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:44.073337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:44.093564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4beb9-13d7-4cf3-9dbb-2e511f9114b3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:44.132543 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605ccab-6bdc-4a1e-83f8-ab09e07b5674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:44.133263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605ccab-6bdc-4a1e-83f8-ab09e07b5674 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:44.133608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605ccab-6bdc-4a1e-83f8-ab09e07b5674 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:44.134202 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605ccab-6bdc-4a1e-83f8-ab09e07b5674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:44.134739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605ccab-6bdc-4a1e-83f8-ab09e07b5674 None admin] RBAC: Adding query filter params (role.id=290368e3589e425e99a9f4dbfe2b3823, user.id=6308242b9a9f49a387e447cab1244c22, scope.system=all) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:44.135108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605ccab-6bdc-4a1e-83f8-ab09e07b5674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:44.136729 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605ccab-6bdc-4a1e-83f8-ab09e07b5674 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:44.142904 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 110/219] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Oct 24 08:34:43 2022] GET /identity/v3/role_assignments?user.id=6308242b9a9f49a387e447cab1244c22&scope.system=all&role.id=290368e3589e425e99a9f4dbfe2b3823 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:44.162808 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de46e7-d3c8-447d-ba81-9094d29952ce None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:44.201385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613177eb-5caa-4763-8519-3a975723e289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:44.202359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613177eb-5caa-4763-8519-3a975723e289 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:44.202883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613177eb-5caa-4763-8519-3a975723e289 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:44.203317 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613177eb-5caa-4763-8519-3a975723e289 None admin] PATH_INFO: `/v3/users/aecf9e7820284aeebad7fefafb35d5d2` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:44.204789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613177eb-5caa-4763-8519-3a975723e289 None admin] RBAC: Authorizing `identity:get_user(user_id=aecf9e7820284aeebad7fefafb35d5d2)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:44.206474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613177eb-5caa-4763-8519-3a975723e289 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:44.208438 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 110/220] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:44 2022] GET /identity/v3/users/aecf9e7820284aeebad7fefafb35d5d2 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:44.218771 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605ccab-6bdc-4a1e-83f8-ab09e07b5674 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:44.301389 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:44.343099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605ccab-6bdc-4a1e-83f8-ab09e07b5674 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:44.377968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541f471-05fd-48a5-acf2-e9bde4129414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:44.378580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541f471-05fd-48a5-acf2-e9bde4129414 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:44.378887 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541f471-05fd-48a5-acf2-e9bde4129414 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:44.379154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541f471-05fd-48a5-acf2-e9bde4129414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:44.445479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541f471-05fd-48a5-acf2-e9bde4129414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:44.446970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541f471-05fd-48a5-acf2-e9bde4129414 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:44.507587 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:44.548062 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 111/221] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:44.553839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613177eb-5caa-4763-8519-3a975723e289 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:44.653672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:44.660692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613177eb-5caa-4763-8519-3a975723e289 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:44.697710 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ce0e9-37d2-4e23-bd2e-3583c4693482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:44.698413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce0e9-37d2-4e23-bd2e-3583c4693482 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:44.698690 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce0e9-37d2-4e23-bd2e-3583c4693482 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:44.698989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce0e9-37d2-4e23-bd2e-3583c4693482 None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:44.700140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce0e9-37d2-4e23-bd2e-3583c4693482 None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:44.701695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce0e9-37d2-4e23-bd2e-3583c4693482 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:44.703664 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 111/222] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:44 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:44.711886 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541f471-05fd-48a5-acf2-e9bde4129414 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:44.817255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541f471-05fd-48a5-acf2-e9bde4129414 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:44.855961 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19054ed-86cf-4730-a38a-9ce0c3370895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:44.856574 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19054ed-86cf-4730-a38a-9ce0c3370895 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:44.856848 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19054ed-86cf-4730-a38a-9ce0c3370895 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:44.857116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19054ed-86cf-4730-a38a-9ce0c3370895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:44.857834 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19054ed-86cf-4730-a38a-9ce0c3370895 None admin] RBAC: Adding query filter params (role.id=b96f3faefe1c453fb20deea4530c43f0, scope.project.id=157d82dd257b4e0b929f245d80d41b62, user.id=aecf9e7820284aeebad7fefafb35d5d2) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:44.858138 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19054ed-86cf-4730-a38a-9ce0c3370895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:44.859597 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19054ed-86cf-4730-a38a-9ce0c3370895 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:44.865577 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 112/223] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:34:44 2022] GET /identity/v3/role_assignments?user.id=aecf9e7820284aeebad7fefafb35d5d2&scope.project.id=157d82dd257b4e0b929f245d80d41b62&role.id=b96f3faefe1c453fb20deea4530c43f0 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:46.058646 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff37a6d-6c63-44e0-8c54-bcfc292d9c4b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.059369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff37a6d-6c63-44e0-8c54-bcfc292d9c4b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.059369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff37a6d-6c63-44e0-8c54-bcfc292d9c4b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.060922 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 112/224] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:46.066933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeefaa-d7f9-475c-849f-d23406cf6d45 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.067473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeefaa-d7f9-475c-849f-d23406cf6d45 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.067473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeefaa-d7f9-475c-849f-d23406cf6d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.087296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-debeefaa-d7f9-475c-849f-d23406cf6d45 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:46.088486 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-debeefaa-d7f9-475c-849f-d23406cf6d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:46.110683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:46.160065 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 113/225] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:46.172706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220caa0d-38cd-42e3-8f11-fa1755f62135 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.172963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220caa0d-38cd-42e3-8f11-fa1755f62135 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.173248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220caa0d-38cd-42e3-8f11-fa1755f62135 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.174734 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 113/226] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:46.179799 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a69d4-8827-4156-b10d-539a1f080b65 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.180087 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a69d4-8827-4156-b10d-539a1f080b65 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.180327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a69d4-8827-4156-b10d-539a1f080b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.198785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-719a69d4-8827-4156-b10d-539a1f080b65 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:46.200020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-719a69d4-8827-4156-b10d-539a1f080b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:46.271108 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 114/227] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:46.386198 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf97a61-a24a-4e6b-935f-ff11d7c8ded9 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.386535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf97a61-a24a-4e6b-935f-ff11d7c8ded9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.386766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf97a61-a24a-4e6b-935f-ff11d7c8ded9 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.388295 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 114/228] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:46.394039 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd700825-b9c7-4bed-ad9b-ffd0231308a0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.394329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd700825-b9c7-4bed-ad9b-ffd0231308a0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.394598 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd700825-b9c7-4bed-ad9b-ffd0231308a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.414057 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cd700825-b9c7-4bed-ad9b-ffd0231308a0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:46.415259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cd700825-b9c7-4bed-ad9b-ffd0231308a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:46.428941 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf97a61-a24a-4e6b-935f-ff11d7c8ded9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:46.523125 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 115/229] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:46.538082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd700825-b9c7-4bed-ad9b-ffd0231308a0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:46.557055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:46.571344 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf97a61-a24a-4e6b-935f-ff11d7c8ded9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:46.605273 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c78a5-1702-46cc-a0cd-635ec53723df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:46.606024 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c78a5-1702-46cc-a0cd-635ec53723df None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.606305 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c78a5-1702-46cc-a0cd-635ec53723df None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.606576 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c78a5-1702-46cc-a0cd-635ec53723df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.617058 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c78a5-1702-46cc-a0cd-635ec53723df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-8d5c78a5-1702-46cc-a0cd-635ec53723df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:46.622525 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:46.623459 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:46.624639 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 115/230] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:46.629586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c78a5-1702-46cc-a0cd-635ec53723df None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:46.654188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd700825-b9c7-4bed-ad9b-ffd0231308a0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:46.654345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:46.699442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f86ae6-771a-4d29-8d33-6020d9b3fa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:46.700164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f86ae6-771a-4d29-8d33-6020d9b3fa3f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.700450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f86ae6-771a-4d29-8d33-6020d9b3fa3f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.700719 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f86ae6-771a-4d29-8d33-6020d9b3fa3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.701860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f86ae6-771a-4d29-8d33-6020d9b3fa3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:46.703487 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f86ae6-771a-4d29-8d33-6020d9b3fa3f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:46.709928 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 116/231] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:46.710744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f86ae6-771a-4d29-8d33-6020d9b3fa3f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:46.751807 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c78a5-1702-46cc-a0cd-635ec53723df None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:46.788012 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1db06-6d07-4a52-8ca9-dbd039f058ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:46.788694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1db06-6d07-4a52-8ca9-dbd039f058ac None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.788978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1db06-6d07-4a52-8ca9-dbd039f058ac None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.789250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1db06-6d07-4a52-8ca9-dbd039f058ac None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.789913 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1db06-6d07-4a52-8ca9-dbd039f058ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:46.790219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1db06-6d07-4a52-8ca9-dbd039f058ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:46.791645 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1db06-6d07-4a52-8ca9-dbd039f058ac None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:46.799795 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 116/232] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:46.800570 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1db06-6d07-4a52-8ca9-dbd039f058ac None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:46.801015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:46.818913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f86ae6-771a-4d29-8d33-6020d9b3fa3f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:46.851193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49481e2d-1bb3-496d-9c6b-75ab1ed7455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:46.851792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49481e2d-1bb3-496d-9c6b-75ab1ed7455f None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.852064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49481e2d-1bb3-496d-9c6b-75ab1ed7455f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.852330 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49481e2d-1bb3-496d-9c6b-75ab1ed7455f None admin] PATH_INFO: `/v3/groups` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.852952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49481e2d-1bb3-496d-9c6b-75ab1ed7455f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:46.854382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49481e2d-1bb3-496d-9c6b-75ab1ed7455f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:46.870175 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 117/233] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Oct 24 08:34:46 2022] POST /identity/v3/groups => generated 229 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:46.870987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49481e2d-1bb3-496d-9c6b-75ab1ed7455f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:46.892908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:46.913820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1db06-6d07-4a52-8ca9-dbd039f058ac None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:46.951170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09cde9-6e43-4ec2-90e7-0cd54fa15c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:46.951835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09cde9-6e43-4ec2-90e7-0cd54fa15c68 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:46.952120 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09cde9-6e43-4ec2-90e7-0cd54fa15c68 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:46.952420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09cde9-6e43-4ec2-90e7-0cd54fa15c68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:46.967303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09cde9-6e43-4ec2-90e7-0cd54fa15c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-da09cde9-6e43-4ec2-90e7-0cd54fa15c68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:46.972825 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:46.973790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:46.974622 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 117/234] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:46.979999 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09cde9-6e43-4ec2-90e7-0cd54fa15c68 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:46.991520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49481e2d-1bb3-496d-9c6b-75ab1ed7455f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:47.030608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d3426-d81d-4b2f-baa4-277af61d95ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:47.031311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d3426-d81d-4b2f-baa4-277af61d95ce None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:47.031599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d3426-d81d-4b2f-baa4-277af61d95ce None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:47.031877 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d3426-d81d-4b2f-baa4-277af61d95ce None admin] PATH_INFO: `/v3/users/f88771844c0448cf9581afa05bbe9309` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:47.048980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d3426-d81d-4b2f-baa4-277af61d95ce None admin] RBAC: Authorizing `identity:get_user(user_id=f88771844c0448cf9581afa05bbe9309)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:47.049269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d3426-d81d-4b2f-baa4-277af61d95ce None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:47.055105 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 118/235] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:46 2022] GET /identity/v3/users/f88771844c0448cf9581afa05bbe9309 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:47.056701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d3426-d81d-4b2f-baa4-277af61d95ce None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:47.059529 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:47.094437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09cde9-6e43-4ec2-90e7-0cd54fa15c68 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:47.128977 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec584-8b73-4b63-a193-5b07bb79c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:47.129886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec584-8b73-4b63-a193-5b07bb79c3e7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:47.130359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec584-8b73-4b63-a193-5b07bb79c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:47.130881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec584-8b73-4b63-a193-5b07bb79c3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:47.131753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ec584-8b73-4b63-a193-5b07bb79c3e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:47.132054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ec584-8b73-4b63-a193-5b07bb79c3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:47.133495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ec584-8b73-4b63-a193-5b07bb79c3e7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:47.141840 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 118/236] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:47.147197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec584-8b73-4b63-a193-5b07bb79c3e7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:47.156495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:47.166814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d3426-d81d-4b2f-baa4-277af61d95ce None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:47.204111 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba48e7-dd73-4b61-a2fe-aa9192464ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:47.204703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba48e7-dd73-4b61-a2fe-aa9192464ebd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:47.204979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba48e7-dd73-4b61-a2fe-aa9192464ebd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:47.205259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba48e7-dd73-4b61-a2fe-aa9192464ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:47.205966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba48e7-dd73-4b61-a2fe-aa9192464ebd None admin] RBAC: Adding query filter params (role.id=b96f3faefe1c453fb20deea4530c43f0, user.id=f88771844c0448cf9581afa05bbe9309, scope.system=all) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:47.206280 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba48e7-dd73-4b61-a2fe-aa9192464ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:47.207738 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba48e7-dd73-4b61-a2fe-aa9192464ebd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:47.213011 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 119/237] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Oct 24 08:34:47 2022] GET /identity/v3/role_assignments?user.id=f88771844c0448cf9581afa05bbe9309&scope.system=all&role.id=b96f3faefe1c453fb20deea4530c43f0 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:47.253735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec584-8b73-4b63-a193-5b07bb79c3e7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:47.298062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26349c6-647a-4678-8fa9-0c89cb9a0a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:47.298849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26349c6-647a-4678-8fa9-0c89cb9a0a9f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:47.299128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26349c6-647a-4678-8fa9-0c89cb9a0a9f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:47.299395 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26349c6-647a-4678-8fa9-0c89cb9a0a9f None admin] PATH_INFO: `/v3/users/6308242b9a9f49a387e447cab1244c22` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:47.300585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26349c6-647a-4678-8fa9-0c89cb9a0a9f None admin] RBAC: Authorizing `identity:get_user(user_id=6308242b9a9f49a387e447cab1244c22)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:47.302061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26349c6-647a-4678-8fa9-0c89cb9a0a9f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:47.303955 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 119/238] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:47 2022] GET /identity/v3/users/6308242b9a9f49a387e447cab1244c22 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:47.309714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba48e7-dd73-4b61-a2fe-aa9192464ebd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:47.383147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:47.420528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba48e7-dd73-4b61-a2fe-aa9192464ebd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:47.455100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef8577-a16a-4b76-b0e9-9697f4f40eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:47.455749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef8577-a16a-4b76-b0e9-9697f4f40eb7 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:47.456024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef8577-a16a-4b76-b0e9-9697f4f40eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:47.456295 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef8577-a16a-4b76-b0e9-9697f4f40eb7 None admin] PATH_INFO: `/v3/system/users/6308242b9a9f49a387e447cab1244c22/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:47.457684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef8577-a16a-4b76-b0e9-9697f4f40eb7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6308242b9a9f49a387e447cab1244c22, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:47.459100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef8577-a16a-4b76-b0e9-9697f4f40eb7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:47.469608 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 120/239] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Oct 24 08:34:47 2022] PUT /identity/v3/system/users/6308242b9a9f49a387e447cab1244c22/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:49.026127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053c01e-b961-4c59-b9c5-cbfb055b573c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:49.026675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053c01e-b961-4c59-b9c5-cbfb055b573c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:49.026675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053c01e-b961-4c59-b9c5-cbfb055b573c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:49.028296 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 120/240] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:49.033424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d796b1-6856-47d2-8ff0-2160cf27c3dd None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:49.033838 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d796b1-6856-47d2-8ff0-2160cf27c3dd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:49.033929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d796b1-6856-47d2-8ff0-2160cf27c3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:49.052551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-21d796b1-6856-47d2-8ff0-2160cf27c3dd None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:49.053702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-21d796b1-6856-47d2-8ff0-2160cf27c3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:49.120277 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 121/241] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:49.344384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23ac9d-7e23-4ef8-9792-7a5b1e9efd07 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:49.344720 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23ac9d-7e23-4ef8-9792-7a5b1e9efd07 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:49.344949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23ac9d-7e23-4ef8-9792-7a5b1e9efd07 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:49.346482 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 121/242] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:49.351588 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2147-1118-4c4e-9982-a75b3dea57b8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:49.351879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2147-1118-4c4e-9982-a75b3dea57b8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:49.352154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd2147-1118-4c4e-9982-a75b3dea57b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:49.370430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-92bd2147-1118-4c4e-9982-a75b3dea57b8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:49.371705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-92bd2147-1118-4c4e-9982-a75b3dea57b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:49.380107 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23ac9d-7e23-4ef8-9792-7a5b1e9efd07 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:49.432453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:49.447322 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 122/243] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:49.473632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:49.503478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23ac9d-7e23-4ef8-9792-7a5b1e9efd07 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:49.540102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b4045-e189-4d2e-8ee8-4b73d9f49150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:49.540651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b4045-e189-4d2e-8ee8-4b73d9f49150 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:49.540924 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b4045-e189-4d2e-8ee8-4b73d9f49150 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:49.541194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b4045-e189-4d2e-8ee8-4b73d9f49150 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:49.542283 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b4045-e189-4d2e-8ee8-4b73d9f49150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:49.543825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b4045-e189-4d2e-8ee8-4b73d9f49150 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:49.545547 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 122/244] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:34:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:49.551056 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2147-1118-4c4e-9982-a75b3dea57b8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:49.606537 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571bf13-8788-4425-8224-33a80033c39a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:49.606845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571bf13-8788-4425-8224-33a80033c39a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:49.607117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571bf13-8788-4425-8224-33a80033c39a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:49.608476 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 123/245] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:49.614136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5b81c-5e25-47f4-896c-701b8317d8d5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:49.614546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5b81c-5e25-47f4-896c-701b8317d8d5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:49.614856 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5b81c-5e25-47f4-896c-701b8317d8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:49.633408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6fb5b81c-5e25-47f4-896c-701b8317d8d5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:49.634601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6fb5b81c-5e25-47f4-896c-701b8317d8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:49.652067 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd2147-1118-4c4e-9982-a75b3dea57b8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:49.685605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224b662-dfea-4dd3-baae-faba667d9ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:49.686237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224b662-dfea-4dd3-baae-faba667d9ebc None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:49.686464 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224b662-dfea-4dd3-baae-faba667d9ebc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:49.686773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224b662-dfea-4dd3-baae-faba667d9ebc None admin] PATH_INFO: `/v3/groups` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:49.687415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224b662-dfea-4dd3-baae-faba667d9ebc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:49.688858 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224b662-dfea-4dd3-baae-faba667d9ebc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:49.705541 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 123/246] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Oct 24 08:34:49 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:34:49.705732 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 124/247] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:49.717413 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224b662-dfea-4dd3-baae-faba667d9ebc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:49.776020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:49.833651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224b662-dfea-4dd3-baae-faba667d9ebc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:49.870303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb663916-5190-4cc5-bdcd-15190ea6239f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:49.870968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663916-5190-4cc5-bdcd-15190ea6239f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:49.871246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663916-5190-4cc5-bdcd-15190ea6239f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:49.871590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663916-5190-4cc5-bdcd-15190ea6239f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:49.883732 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb663916-5190-4cc5-bdcd-15190ea6239f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-fb663916-5190-4cc5-bdcd-15190ea6239f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:49.889066 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:49.890044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:49.891028 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 124/248] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:49.895803 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5b81c-5e25-47f4-896c-701b8317d8d5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:49.967542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:49.972065 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb663916-5190-4cc5-bdcd-15190ea6239f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:50.002744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5b81c-5e25-47f4-896c-701b8317d8d5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:50.039010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bc870-1269-4a3e-935a-8f4f7ee8942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:50.039656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bc870-1269-4a3e-935a-8f4f7ee8942f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:50.039890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bc870-1269-4a3e-935a-8f4f7ee8942f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:50.040154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bc870-1269-4a3e-935a-8f4f7ee8942f None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:50.040818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bc870-1269-4a3e-935a-8f4f7ee8942f None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:50.041110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bc870-1269-4a3e-935a-8f4f7ee8942f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:50.042505 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bc870-1269-4a3e-935a-8f4f7ee8942f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:50.050949 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 125/249] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:50.056409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bc870-1269-4a3e-935a-8f4f7ee8942f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:50.096442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb663916-5190-4cc5-bdcd-15190ea6239f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:50.108485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:50.138606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe039bc9-4c1e-4d24-98ac-526fbd2b9906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:50.139550 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe039bc9-4c1e-4d24-98ac-526fbd2b9906 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:50.139828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe039bc9-4c1e-4d24-98ac-526fbd2b9906 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:50.140098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe039bc9-4c1e-4d24-98ac-526fbd2b9906 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:50.154285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe039bc9-4c1e-4d24-98ac-526fbd2b9906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-fe039bc9-4c1e-4d24-98ac-526fbd2b9906 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:50.160686 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:50.161978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:50.162889 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 125/250] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:34:50.168506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe039bc9-4c1e-4d24-98ac-526fbd2b9906 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:50.177667 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bc870-1269-4a3e-935a-8f4f7ee8942f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:50.214805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be670e-e8bc-487d-9f12-488fcceb26e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:50.215491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be670e-e8bc-487d-9f12-488fcceb26e1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:50.215770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be670e-e8bc-487d-9f12-488fcceb26e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:50.216040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be670e-e8bc-487d-9f12-488fcceb26e1 None admin] PATH_INFO: `/v3/users/f88771844c0448cf9581afa05bbe9309` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:50.217215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be670e-e8bc-487d-9f12-488fcceb26e1 None admin] RBAC: Authorizing `identity:get_user(user_id=f88771844c0448cf9581afa05bbe9309)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:50.218670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be670e-e8bc-487d-9f12-488fcceb26e1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:50.220516 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 126/251] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:50 2022] GET /identity/v3/users/f88771844c0448cf9581afa05bbe9309 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:50.226249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be670e-e8bc-487d-9f12-488fcceb26e1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:50.229429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:50.293840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe039bc9-4c1e-4d24-98ac-526fbd2b9906 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:50.330017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ea7dd-9bc7-4b52-83b0-b9de936097fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:50.330996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ea7dd-9bc7-4b52-83b0-b9de936097fc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:50.331468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ea7dd-9bc7-4b52-83b0-b9de936097fc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:50.331973 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ea7dd-9bc7-4b52-83b0-b9de936097fc None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:50.332852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ea7dd-9bc7-4b52-83b0-b9de936097fc None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:50.333342 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ea7dd-9bc7-4b52-83b0-b9de936097fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:50.335169 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ea7dd-9bc7-4b52-83b0-b9de936097fc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:50.344378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be670e-e8bc-487d-9f12-488fcceb26e1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:50.349225 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 126/252] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:50.350628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ea7dd-9bc7-4b52-83b0-b9de936097fc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:50.391161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:50.393927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cef7be-9cc4-4ab7-ace5-5a73038934c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:50.395207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cef7be-9cc4-4ab7-ace5-5a73038934c2 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:50.396530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cef7be-9cc4-4ab7-ace5-5a73038934c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:50.397016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cef7be-9cc4-4ab7-ace5-5a73038934c2 None admin] PATH_INFO: `/v3/system/users/f88771844c0448cf9581afa05bbe9309/roles/b96f3faefe1c453fb20deea4530c43f0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:50.398739 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cef7be-9cc4-4ab7-ace5-5a73038934c2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f88771844c0448cf9581afa05bbe9309, role_id=b96f3faefe1c453fb20deea4530c43f0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:50.400357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cef7be-9cc4-4ab7-ace5-5a73038934c2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:50.412901 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 127/253] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Oct 24 08:34:50 2022] PUT /identity/v3/system/users/f88771844c0448cf9581afa05bbe9309/roles/b96f3faefe1c453fb20deea4530c43f0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:50.475048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ea7dd-9bc7-4b52-83b0-b9de936097fc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:50.514580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d98046-2ec9-4342-961d-fa45ff439785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:50.515313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d98046-2ec9-4342-961d-fa45ff439785 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:50.515592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d98046-2ec9-4342-961d-fa45ff439785 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:50.515862 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d98046-2ec9-4342-961d-fa45ff439785 None admin] PATH_INFO: `/v3/users/6308242b9a9f49a387e447cab1244c22` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:50.517086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d98046-2ec9-4342-961d-fa45ff439785 None admin] RBAC: Authorizing `identity:get_user(user_id=6308242b9a9f49a387e447cab1244c22)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:50.518552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d98046-2ec9-4342-961d-fa45ff439785 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:50.525297 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 127/254] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:50 2022] GET /identity/v3/users/6308242b9a9f49a387e447cab1244c22 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:50.527084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cef7be-9cc4-4ab7-ace5-5a73038934c2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:50.623014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:50.669995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cef7be-9cc4-4ab7-ace5-5a73038934c2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:50.704640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce463025-2d0d-44b2-89b3-ba68d5791e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:50.705536 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce463025-2d0d-44b2-89b3-ba68d5791e91 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:50.705954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce463025-2d0d-44b2-89b3-ba68d5791e91 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:50.706361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce463025-2d0d-44b2-89b3-ba68d5791e91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:50.707256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce463025-2d0d-44b2-89b3-ba68d5791e91 None admin] RBAC: Adding query filter params (role.id=290368e3589e425e99a9f4dbfe2b3823, user.id=6308242b9a9f49a387e447cab1244c22, scope.system=all) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:50.707699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce463025-2d0d-44b2-89b3-ba68d5791e91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:50.709306 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce463025-2d0d-44b2-89b3-ba68d5791e91 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:50.723699 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 128/255] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Oct 24 08:34:50 2022] GET /identity/v3/role_assignments?user.id=6308242b9a9f49a387e447cab1244c22&scope.system=all&role.id=290368e3589e425e99a9f4dbfe2b3823 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:52.122435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ef245-9562-49ab-b27c-a70f8eeff8e5 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:52.123318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ef245-9562-49ab-b27c-a70f8eeff8e5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:52.123832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ef245-9562-49ab-b27c-a70f8eeff8e5 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:52.125662 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 128/256] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:52.130656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a48fcb-49b7-414d-869f-3d1b8cc45591 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:52.131493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a48fcb-49b7-414d-869f-3d1b8cc45591 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:52.131978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a48fcb-49b7-414d-869f-3d1b8cc45591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:52.151555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d5a48fcb-49b7-414d-869f-3d1b8cc45591 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:52.153140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d5a48fcb-49b7-414d-869f-3d1b8cc45591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:52.225453 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 129/257] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 08:34:52.586086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f45015-c075-4f27-9da9-fa7753d65e5c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:52.586755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f45015-c075-4f27-9da9-fa7753d65e5c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:52.587256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f45015-c075-4f27-9da9-fa7753d65e5c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:52.589086 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 129/258] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:52.594432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77289d0-2373-442b-9aaf-1f6e2304a7d7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:52.594989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77289d0-2373-442b-9aaf-1f6e2304a7d7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:52.595475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77289d0-2373-442b-9aaf-1f6e2304a7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:52.617615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c77289d0-2373-442b-9aaf-1f6e2304a7d7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:52.619140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c77289d0-2373-442b-9aaf-1f6e2304a7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:52.630440 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 130/259] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 08:34:52.658394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f45015-c075-4f27-9da9-fa7753d65e5c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:52.677036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f45015-c075-4f27-9da9-fa7753d65e5c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:52.681594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bc8f3-7d04-475d-a140-dbd885322c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:52.682447 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bc8f3-7d04-475d-a140-dbd885322c0c admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:52.682943 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bc8f3-7d04-475d-a140-dbd885322c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:52.683423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bc8f3-7d04-475d-a140-dbd885322c0c admin admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:52.695535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bc8f3-7d04-475d-a140-dbd885322c0c admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-cc0bc8f3-7d04-475d-a140-dbd885322c0c admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:52.701180 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:52.702067 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:52.702906 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:52.705595 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 130/260] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Oct 24 08:34:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:52.711008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77289d0-2373-442b-9aaf-1f6e2304a7d7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:52.721083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77289d0-2373-442b-9aaf-1f6e2304a7d7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:52.725882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:52.730762 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:52.731293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:52.731782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:52.732657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:52.733169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:52.735216 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING py.warnings [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:34:52.735216 dv-cinder-master-3-smb devstack@keystone.service[87873]: warnings.warn(msg) Oct 24 08:34:52.735216 dv-cinder-master-3-smb devstack@keystone.service[87873]: Oct 24 08:34:52.736453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:52.746777 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 131/261] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:34:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:52.752023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bc8f3-7d04-475d-a140-dbd885322c0c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:52.762854 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bc8f3-7d04-475d-a140-dbd885322c0c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:52.767903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9965e-cbe1-4dd7-989f-520108f277ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:52.768920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9965e-cbe1-4dd7-989f-520108f277ba admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:52.769459 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9965e-cbe1-4dd7-989f-520108f277ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:52.769918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9965e-cbe1-4dd7-989f-520108f277ba admin admin] PATH_INFO: `/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:52.779695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9965e-cbe1-4dd7-989f-520108f277ba admin admin] RBAC: Authorizing `identity:get_group(group_id=182e8ffbac2d4f89b7c20afe3b523cf0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:52.781888 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING py.warnings [None req-ebe9965e-cbe1-4dd7-989f-520108f277ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:34:52.781888 dv-cinder-master-3-smb devstack@keystone.service[87872]: warnings.warn(msg) Oct 24 08:34:52.781888 dv-cinder-master-3-smb devstack@keystone.service[87872]: Oct 24 08:34:52.783566 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9965e-cbe1-4dd7-989f-520108f277ba admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:52.785512 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 131/262] 10.222.0.26 () {56 vars in 1201 bytes} [Mon Oct 24 08:34:52 2022] GET /identity/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0 => generated 226 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:52.791564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:52.802288 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c5fb-3b05-485e-96ae-de5bdc23e2b8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:52.811406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112674c-9402-4b63-8de1-98ac0c06e2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:52.812377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112674c-9402-4b63-8de1-98ac0c06e2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:52.812847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112674c-9402-4b63-8de1-98ac0c06e2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:52.813357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112674c-9402-4b63-8de1-98ac0c06e2d1 admin admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:52.814784 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112674c-9402-4b63-8de1-98ac0c06e2d1 admin admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:52.817293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112674c-9402-4b63-8de1-98ac0c06e2d1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:52.819296 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 132/263] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Oct 24 08:34:52 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:52.825118 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9965e-cbe1-4dd7-989f-520108f277ba admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:52.835945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9965e-cbe1-4dd7-989f-520108f277ba admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:52.840889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14507a92-bd14-4e76-af1d-4504a4df0913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:52.841836 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14507a92-bd14-4e76-af1d-4504a4df0913 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:52.842370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14507a92-bd14-4e76-af1d-4504a4df0913 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:52.842976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14507a92-bd14-4e76-af1d-4504a4df0913 admin admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/groups/182e8ffbac2d4f89b7c20afe3b523cf0/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:52.845107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14507a92-bd14-4e76-af1d-4504a4df0913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538dc9f6865b4b1a932b75e3be851bb7, group_id=182e8ffbac2d4f89b7c20afe3b523cf0, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:52.846435 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING py.warnings [None req-14507a92-bd14-4e76-af1d-4504a4df0913 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:34:52.846435 dv-cinder-master-3-smb devstack@keystone.service[87872]: warnings.warn(msg) Oct 24 08:34:52.846435 dv-cinder-master-3-smb devstack@keystone.service[87872]: Oct 24 08:34:52.848858 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14507a92-bd14-4e76-af1d-4504a4df0913 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:52.873240 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 132/264] 10.222.0.26 () {58 vars in 1463 bytes} [Mon Oct 24 08:34:52 2022] PUT /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/groups/182e8ffbac2d4f89b7c20afe3b523cf0/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:53.039019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1f8dc-7ab3-498c-852b-d1c62e07beb4 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.039701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1f8dc-7ab3-498c-852b-d1c62e07beb4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.040236 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1f8dc-7ab3-498c-852b-d1c62e07beb4 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.042073 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 133/265] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:53.047904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212d8e0-0e49-4dab-b476-c35c59f91581 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.048427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212d8e0-0e49-4dab-b476-c35c59f91581 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.048916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212d8e0-0e49-4dab-b476-c35c59f91581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.074038 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3212d8e0-0e49-4dab-b476-c35c59f91581 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:53.075740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3212d8e0-0e49-4dab-b476-c35c59f91581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:53.112265 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1f8dc-7ab3-498c-852b-d1c62e07beb4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.173409 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 133/266] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 08:34:53.220213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:53.228625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1f8dc-7ab3-498c-852b-d1c62e07beb4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.236272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fbf99-6c79-41fc-a110-2a2c90c794ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.236935 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fbf99-6c79-41fc-a110-2a2c90c794ae admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.237217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fbf99-6c79-41fc-a110-2a2c90c794ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.237493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fbf99-6c79-41fc-a110-2a2c90c794ae admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.249780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fbf99-6c79-41fc-a110-2a2c90c794ae admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-687fbf99-6c79-41fc-a110-2a2c90c794ae admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:53.255128 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:53.256022 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:53.256834 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 134/267] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Oct 24 08:34:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:34:53.262832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212d8e0-0e49-4dab-b476-c35c59f91581 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.293193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212d8e0-0e49-4dab-b476-c35c59f91581 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.304578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.305213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.305495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.305763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.306444 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:53.306760 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:53.307791 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING py.warnings [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:34:53.307791 dv-cinder-master-3-smb devstack@keystone.service[87872]: warnings.warn(msg) Oct 24 08:34:53.307791 dv-cinder-master-3-smb devstack@keystone.service[87872]: Oct 24 08:34:53.308701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:53.325264 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 134/268] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Oct 24 08:34:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:53.330975 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fbf99-6c79-41fc-a110-2a2c90c794ae admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.344313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fbf99-6c79-41fc-a110-2a2c90c794ae admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.348669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50f889-d78d-45b3-a9b1-bba278a59444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.349318 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50f889-d78d-45b3-a9b1-bba278a59444 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.349601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50f889-d78d-45b3-a9b1-bba278a59444 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.349867 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50f889-d78d-45b3-a9b1-bba278a59444 admin admin] PATH_INFO: `/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.355451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50f889-d78d-45b3-a9b1-bba278a59444 admin admin] RBAC: Authorizing `identity:get_group(group_id=182e8ffbac2d4f89b7c20afe3b523cf0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:53.356569 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING py.warnings [None req-6a50f889-d78d-45b3-a9b1-bba278a59444 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:34:53.356569 dv-cinder-master-3-smb devstack@keystone.service[87873]: warnings.warn(msg) Oct 24 08:34:53.356569 dv-cinder-master-3-smb devstack@keystone.service[87873]: Oct 24 08:34:53.357799 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50f889-d78d-45b3-a9b1-bba278a59444 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:53.362907 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 135/269] 10.222.0.26 () {56 vars in 1201 bytes} [Mon Oct 24 08:34:53 2022] GET /identity/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0 => generated 226 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:53.364887 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.376393 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:53.382991 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42e1a7-7562-4736-8ea9-0724c14d8618 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.388961 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e89a07-71e6-4b42-8d64-0392fa5bb469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.389647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e89a07-71e6-4b42-8d64-0392fa5bb469 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.389995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e89a07-71e6-4b42-8d64-0392fa5bb469 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.390394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e89a07-71e6-4b42-8d64-0392fa5bb469 admin admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.394213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e89a07-71e6-4b42-8d64-0392fa5bb469 admin admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:53.396712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e89a07-71e6-4b42-8d64-0392fa5bb469 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:53.403097 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 135/270] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Oct 24 08:34:53 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:53.408929 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50f889-d78d-45b3-a9b1-bba278a59444 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.422170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50f889-d78d-45b3-a9b1-bba278a59444 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.426536 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a25cae-b8e2-4f84-87ba-48d2d72a57a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.427278 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a25cae-b8e2-4f84-87ba-48d2d72a57a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.427562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a25cae-b8e2-4f84-87ba-48d2d72a57a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.427839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a25cae-b8e2-4f84-87ba-48d2d72a57a6 admin admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/groups/182e8ffbac2d4f89b7c20afe3b523cf0/roles/5cd206a783e44ac2829c7661b34bb6a7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.429962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a25cae-b8e2-4f84-87ba-48d2d72a57a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538dc9f6865b4b1a932b75e3be851bb7, group_id=182e8ffbac2d4f89b7c20afe3b523cf0, role_id=5cd206a783e44ac2829c7661b34bb6a7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:53.431028 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING py.warnings [None req-65a25cae-b8e2-4f84-87ba-48d2d72a57a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:34:53.431028 dv-cinder-master-3-smb devstack@keystone.service[87873]: warnings.warn(msg) Oct 24 08:34:53.431028 dv-cinder-master-3-smb devstack@keystone.service[87873]: Oct 24 08:34:53.432980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a25cae-b8e2-4f84-87ba-48d2d72a57a6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:53.464177 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 136/271] 10.222.0.26 () {58 vars in 1463 bytes} [Mon Oct 24 08:34:53 2022] PUT /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7/groups/182e8ffbac2d4f89b7c20afe3b523cf0/roles/5cd206a783e44ac2829c7661b34bb6a7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:53.632580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e89a07-71e6-4b42-8d64-0392fa5bb469 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.643125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e89a07-71e6-4b42-8d64-0392fa5bb469 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.711510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28325608-f8d0-4d90-9fa3-ab8eedb743be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.712176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28325608-f8d0-4d90-9fa3-ab8eedb743be admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.712453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28325608-f8d0-4d90-9fa3-ab8eedb743be admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.712723 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28325608-f8d0-4d90-9fa3-ab8eedb743be admin admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.725987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28325608-f8d0-4d90-9fa3-ab8eedb743be admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-28325608-f8d0-4d90-9fa3-ab8eedb743be admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:53.733076 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:53.734120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:53.734956 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:53.738936 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 136/272] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Oct 24 08:34:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 103 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:53.740306 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a25cae-b8e2-4f84-87ba-48d2d72a57a6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.753435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a25cae-b8e2-4f84-87ba-48d2d72a57a6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.758049 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1064773-00e8-4cb2-8fef-1043819f168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.758697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1064773-00e8-4cb2-8fef-1043819f168b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.759008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1064773-00e8-4cb2-8fef-1043819f168b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.759281 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1064773-00e8-4cb2-8fef-1043819f168b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.759969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1064773-00e8-4cb2-8fef-1043819f168b admin admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:53.760272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1064773-00e8-4cb2-8fef-1043819f168b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:53.761977 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1064773-00e8-4cb2-8fef-1043819f168b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:53.775105 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 137/273] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:34:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:53.781240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28325608-f8d0-4d90-9fa3-ab8eedb743be admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.792151 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28325608-f8d0-4d90-9fa3-ab8eedb743be admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.796727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceacbd-d884-4e21-ad1a-f267e7ee0c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.797381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceacbd-d884-4e21-ad1a-f267e7ee0c64 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.797659 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceacbd-d884-4e21-ad1a-f267e7ee0c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.797932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceacbd-d884-4e21-ad1a-f267e7ee0c64 admin admin] PATH_INFO: `/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.803324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceacbd-d884-4e21-ad1a-f267e7ee0c64 admin admin] RBAC: Authorizing `identity:get_group(group_id=182e8ffbac2d4f89b7c20afe3b523cf0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:53.805360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceacbd-d884-4e21-ad1a-f267e7ee0c64 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:53.807074 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 137/274] 10.222.0.26 () {56 vars in 1201 bytes} [Mon Oct 24 08:34:53 2022] GET /identity/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0 => generated 226 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:53.812929 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1064773-00e8-4cb2-8fef-1043819f168b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.822955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1064773-00e8-4cb2-8fef-1043819f168b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.827466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff290c55-5098-497c-aef0-b39cf75cf7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.828118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290c55-5098-497c-aef0-b39cf75cf7db admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.828389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290c55-5098-497c-aef0-b39cf75cf7db admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.828663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290c55-5098-497c-aef0-b39cf75cf7db admin admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.829801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff290c55-5098-497c-aef0-b39cf75cf7db admin admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:53.831912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff290c55-5098-497c-aef0-b39cf75cf7db admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:53.833756 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 138/275] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Oct 24 08:34:53 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:53.839396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceacbd-d884-4e21-ad1a-f267e7ee0c64 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:53.854303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceacbd-d884-4e21-ad1a-f267e7ee0c64 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:53.858916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa80b54-74c6-4b68-9a37-9a0f6e0360c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:53.859590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa80b54-74c6-4b68-9a37-9a0f6e0360c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.859863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa80b54-74c6-4b68-9a37-9a0f6e0360c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.860131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa80b54-74c6-4b68-9a37-9a0f6e0360c3 admin admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62/groups/182e8ffbac2d4f89b7c20afe3b523cf0/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.862004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa80b54-74c6-4b68-9a37-9a0f6e0360c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157d82dd257b4e0b929f245d80d41b62, group_id=182e8ffbac2d4f89b7c20afe3b523cf0, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:53.864698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa80b54-74c6-4b68-9a37-9a0f6e0360c3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:53.894486 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 138/276] 10.222.0.26 () {58 vars in 1463 bytes} [Mon Oct 24 08:34:53 2022] PUT /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62/groups/182e8ffbac2d4f89b7c20afe3b523cf0/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:34:53.944735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e94b44-4935-4ac3-ac24-7dc93023407a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.945056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e94b44-4935-4ac3-ac24-7dc93023407a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.945286 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e94b44-4935-4ac3-ac24-7dc93023407a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.946992 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 139/277] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:53.952378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdc0dd-8766-4ec7-a73c-2de22eeac557 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:53.952656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdc0dd-8766-4ec7-a73c-2de22eeac557 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:53.952915 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdc0dd-8766-4ec7-a73c-2de22eeac557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:53.976292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f6bdc0dd-8766-4ec7-a73c-2de22eeac557 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:53.977514 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f6bdc0dd-8766-4ec7-a73c-2de22eeac557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:54.060504 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 139/278] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:54.435328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e94b44-4935-4ac3-ac24-7dc93023407a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:54.555251 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e94b44-4935-4ac3-ac24-7dc93023407a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:54.591209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e254ea6-46bf-4d00-abdc-dbb7f8b78715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:54.591851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e254ea6-46bf-4d00-abdc-dbb7f8b78715 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:54.592127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e254ea6-46bf-4d00-abdc-dbb7f8b78715 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:54.592407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e254ea6-46bf-4d00-abdc-dbb7f8b78715 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:54.602700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e254ea6-46bf-4d00-abdc-dbb7f8b78715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2e254ea6-46bf-4d00-abdc-dbb7f8b78715 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:54.608037 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:54.608955 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:54.610137 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 140/279] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:54.616034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdc0dd-8766-4ec7-a73c-2de22eeac557 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:54.675342 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:54.685858 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acefc3-3705-4f0b-95fa-43911a3f6dee None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:54.686140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acefc3-3705-4f0b-95fa-43911a3f6dee None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:54.686412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acefc3-3705-4f0b-95fa-43911a3f6dee None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:54.687794 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 141/280] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:54.694633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ba2ba-ca9c-4aae-9dd4-c2be7406f158 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:54.694941 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ba2ba-ca9c-4aae-9dd4-c2be7406f158 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:54.695224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ba2ba-ca9c-4aae-9dd4-c2be7406f158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:54.722750 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-372ba2ba-ca9c-4aae-9dd4-c2be7406f158 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:54.723975 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-372ba2ba-ca9c-4aae-9dd4-c2be7406f158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:54.753738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:54.764473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdc0dd-8766-4ec7-a73c-2de22eeac557 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:54.818636 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 142/281] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 08:34:54.828489 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8349f-3810-4dfa-aa13-4a53e45990e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:54.829102 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8349f-3810-4dfa-aa13-4a53e45990e1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:54.829377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8349f-3810-4dfa-aa13-4a53e45990e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:54.829644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8349f-3810-4dfa-aa13-4a53e45990e1 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:54.830309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8349f-3810-4dfa-aa13-4a53e45990e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:54.830664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8349f-3810-4dfa-aa13-4a53e45990e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:54.832122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8349f-3810-4dfa-aa13-4a53e45990e1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:54.844828 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 140/282] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:54.850848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ba2ba-ca9c-4aae-9dd4-c2be7406f158 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:54.966922 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ba2ba-ca9c-4aae-9dd4-c2be7406f158 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:55.005442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902debbd-2f9f-43c2-aa52-2b160087c758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:55.006160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902debbd-2f9f-43c2-aa52-2b160087c758 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:55.006453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902debbd-2f9f-43c2-aa52-2b160087c758 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:55.006701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902debbd-2f9f-43c2-aa52-2b160087c758 None admin] PATH_INFO: `/v3/users/f88771844c0448cf9581afa05bbe9309` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:55.007847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902debbd-2f9f-43c2-aa52-2b160087c758 None admin] RBAC: Authorizing `identity:get_user(user_id=f88771844c0448cf9581afa05bbe9309)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:55.009352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902debbd-2f9f-43c2-aa52-2b160087c758 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:55.011370 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 143/283] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:34:54 2022] GET /identity/v3/users/f88771844c0448cf9581afa05bbe9309 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:55.016929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8349f-3810-4dfa-aa13-4a53e45990e1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:55.135595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8349f-3810-4dfa-aa13-4a53e45990e1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:55.152237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:55.172142 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b3b70-e129-499a-a5a5-3c29b276b01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:55.172759 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b3b70-e129-499a-a5a5-3c29b276b01d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:55.173036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b3b70-e129-499a-a5a5-3c29b276b01d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:55.173308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b3b70-e129-499a-a5a5-3c29b276b01d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:55.174148 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b3b70-e129-499a-a5a5-3c29b276b01d None admin] RBAC: Adding query filter params (role.id=b96f3faefe1c453fb20deea4530c43f0, user.id=f88771844c0448cf9581afa05bbe9309, scope.system=all) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:55.174456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b3b70-e129-499a-a5a5-3c29b276b01d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:55.175969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b3b70-e129-499a-a5a5-3c29b276b01d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:55.193093 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 141/284] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Oct 24 08:34:55 2022] GET /identity/v3/role_assignments?user.id=f88771844c0448cf9581afa05bbe9309&scope.system=all&role.id=b96f3faefe1c453fb20deea4530c43f0 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:55.387335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902debbd-2f9f-43c2-aa52-2b160087c758 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:55.399462 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902debbd-2f9f-43c2-aa52-2b160087c758 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:55.404036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cabe5-5707-4245-bdab-336f94ef488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:55.404709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cabe5-5707-4245-bdab-336f94ef488f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:55.404983 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cabe5-5707-4245-bdab-336f94ef488f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:55.405252 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cabe5-5707-4245-bdab-336f94ef488f admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:55.419177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cabe5-5707-4245-bdab-336f94ef488f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a82cabe5-5707-4245-bdab-336f94ef488f admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:55.424151 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:55.425106 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:55.425905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:55.427914 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 144/285] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Oct 24 08:34:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:55.436216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b3b70-e129-499a-a5a5-3c29b276b01d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:55.452963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b3b70-e129-499a-a5a5-3c29b276b01d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:55.457452 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd210e26-e254-4697-b70c-7ef975050651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:55.458153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd210e26-e254-4697-b70c-7ef975050651 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:55.458433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd210e26-e254-4697-b70c-7ef975050651 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:55.458701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd210e26-e254-4697-b70c-7ef975050651 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:55.459414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd210e26-e254-4697-b70c-7ef975050651 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:55.459712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd210e26-e254-4697-b70c-7ef975050651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:55.461344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd210e26-e254-4697-b70c-7ef975050651 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:55.475534 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 142/286] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Oct 24 08:34:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:55.482060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cabe5-5707-4245-bdab-336f94ef488f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:55.496102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cabe5-5707-4245-bdab-336f94ef488f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:55.500979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f3693-3d9c-4498-a412-1694ff6074d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:55.501647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f3693-3d9c-4498-a412-1694ff6074d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:55.501981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f3693-3d9c-4498-a412-1694ff6074d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:55.502258 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f3693-3d9c-4498-a412-1694ff6074d5 admin admin] PATH_INFO: `/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:55.503546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f3693-3d9c-4498-a412-1694ff6074d5 admin admin] RBAC: Authorizing `identity:get_group(group_id=182e8ffbac2d4f89b7c20afe3b523cf0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:55.505547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f3693-3d9c-4498-a412-1694ff6074d5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:55.510945 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 145/287] 10.222.0.26 () {56 vars in 1201 bytes} [Mon Oct 24 08:34:55 2022] GET /identity/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0 => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:55.513618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd210e26-e254-4697-b70c-7ef975050651 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:55.525280 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd210e26-e254-4697-b70c-7ef975050651 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:55.530091 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32b45a-597a-4f01-b507-aa0763081259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:55.530829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32b45a-597a-4f01-b507-aa0763081259 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:55.531109 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32b45a-597a-4f01-b507-aa0763081259 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:55.531424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32b45a-597a-4f01-b507-aa0763081259 admin admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:55.532745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32b45a-597a-4f01-b507-aa0763081259 admin admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:55.535041 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32b45a-597a-4f01-b507-aa0763081259 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:55.537666 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 143/288] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Oct 24 08:34:55 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:55.543751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f3693-3d9c-4498-a412-1694ff6074d5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:55.555903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f3693-3d9c-4498-a412-1694ff6074d5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:55.560521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273421f-8bf6-48b8-9841-5bca9a42c674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:55.561245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273421f-8bf6-48b8-9841-5bca9a42c674 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:55.561525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273421f-8bf6-48b8-9841-5bca9a42c674 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:55.561868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273421f-8bf6-48b8-9841-5bca9a42c674 admin admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62/groups/182e8ffbac2d4f89b7c20afe3b523cf0/roles/5cd206a783e44ac2829c7661b34bb6a7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:55.564161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273421f-8bf6-48b8-9841-5bca9a42c674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157d82dd257b4e0b929f245d80d41b62, group_id=182e8ffbac2d4f89b7c20afe3b523cf0, role_id=5cd206a783e44ac2829c7661b34bb6a7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:55.567065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273421f-8bf6-48b8-9841-5bca9a42c674 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:55.593807 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 146/289] 10.222.0.26 () {58 vars in 1463 bytes} [Mon Oct 24 08:34:55 2022] PUT /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62/groups/182e8ffbac2d4f89b7c20afe3b523cf0/roles/5cd206a783e44ac2829c7661b34bb6a7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:55.692634 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf1570-3840-4ffb-bdc9-8a398f8299a9 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:55.693258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf1570-3840-4ffb-bdc9-8a398f8299a9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:55.693258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf1570-3840-4ffb-bdc9-8a398f8299a9 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:55.694676 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 144/290] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:55.701511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddf636-b338-4da9-8f9d-851a7f04abcc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:55.701809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddf636-b338-4da9-8f9d-851a7f04abcc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:55.702063 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddf636-b338-4da9-8f9d-851a7f04abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:55.716560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:55.722562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3bddf636-b338-4da9-8f9d-851a7f04abcc None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:55.724148 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3bddf636-b338-4da9-8f9d-851a7f04abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:55.798874 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 147/291] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 08:34:56.086087 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf1570-3840-4ffb-bdc9-8a398f8299a9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:56.098709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf1570-3840-4ffb-bdc9-8a398f8299a9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:56.103081 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0820f-cbcb-4062-8862-c62086fdd415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:56.103761 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0820f-cbcb-4062-8862-c62086fdd415 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:56.104040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0820f-cbcb-4062-8862-c62086fdd415 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:56.104307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0820f-cbcb-4062-8862-c62086fdd415 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:56.115099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0820f-cbcb-4062-8862-c62086fdd415 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e3f0820f-cbcb-4062-8862-c62086fdd415 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:56.120083 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:56.122182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:34:56.124130 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:56.130778 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 145/292] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Oct 24 08:34:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:56.130985 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddf636-b338-4da9-8f9d-851a7f04abcc None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:56.136896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddf636-b338-4da9-8f9d-851a7f04abcc None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:56.141450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806b12b-3b36-4b2e-98be-ef30b904adaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:56.142117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806b12b-3b36-4b2e-98be-ef30b904adaa admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:56.142397 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806b12b-3b36-4b2e-98be-ef30b904adaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:56.142672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806b12b-3b36-4b2e-98be-ef30b904adaa admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:56.143383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806b12b-3b36-4b2e-98be-ef30b904adaa admin admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:56.143689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806b12b-3b36-4b2e-98be-ef30b904adaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:56.145316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806b12b-3b36-4b2e-98be-ef30b904adaa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:56.154898 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 148/293] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Oct 24 08:34:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:56.162521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0820f-cbcb-4062-8862-c62086fdd415 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:56.175554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0820f-cbcb-4062-8862-c62086fdd415 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:56.180097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60225ef7-83f6-4343-9ef7-6aacda84f0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:56.180726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60225ef7-83f6-4343-9ef7-6aacda84f0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:56.181052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60225ef7-83f6-4343-9ef7-6aacda84f0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:56.181326 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60225ef7-83f6-4343-9ef7-6aacda84f0d9 admin admin] PATH_INFO: `/v3/groups/47495fe0b14a479bbf55b547abe0975d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:56.186663 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60225ef7-83f6-4343-9ef7-6aacda84f0d9 admin admin] RBAC: Authorizing `identity:get_group(group_id=47495fe0b14a479bbf55b547abe0975d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:56.188715 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60225ef7-83f6-4343-9ef7-6aacda84f0d9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:56.190319 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 146/294] 10.222.0.26 () {56 vars in 1201 bytes} [Mon Oct 24 08:34:56 2022] GET /identity/v3/groups/47495fe0b14a479bbf55b547abe0975d => generated 229 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:56.198209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806b12b-3b36-4b2e-98be-ef30b904adaa admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:56.208629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806b12b-3b36-4b2e-98be-ef30b904adaa admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:56.213441 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e4335-50ca-4dbb-b716-ee7ba02c72a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:56.214296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e4335-50ca-4dbb-b716-ee7ba02c72a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:56.214804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e4335-50ca-4dbb-b716-ee7ba02c72a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:56.215353 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e4335-50ca-4dbb-b716-ee7ba02c72a3 admin admin] PATH_INFO: `/v3/projects/d2f48107e8bb4875908f0d6dc2693154` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:56.216685 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e4335-50ca-4dbb-b716-ee7ba02c72a3 admin admin] RBAC: Authorizing `identity:get_project(project_id=d2f48107e8bb4875908f0d6dc2693154)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:56.218861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e4335-50ca-4dbb-b716-ee7ba02c72a3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:56.225593 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 149/295] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Oct 24 08:34:56 2022] GET /identity/v3/projects/d2f48107e8bb4875908f0d6dc2693154 => generated 344 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:56.227164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60225ef7-83f6-4343-9ef7-6aacda84f0d9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:56.240432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60225ef7-83f6-4343-9ef7-6aacda84f0d9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:56.245594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1d01f-6aca-4d69-8e2f-de5a2625f993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:56.246793 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1d01f-6aca-4d69-8e2f-de5a2625f993 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:56.247246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1d01f-6aca-4d69-8e2f-de5a2625f993 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:56.247734 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1d01f-6aca-4d69-8e2f-de5a2625f993 admin admin] PATH_INFO: `/v3/projects/d2f48107e8bb4875908f0d6dc2693154/groups/47495fe0b14a479bbf55b547abe0975d/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:56.250122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1d01f-6aca-4d69-8e2f-de5a2625f993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f48107e8bb4875908f0d6dc2693154, group_id=47495fe0b14a479bbf55b547abe0975d, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:56.253254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1d01f-6aca-4d69-8e2f-de5a2625f993 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:56.291008 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 147/296] 10.222.0.26 () {58 vars in 1463 bytes} [Mon Oct 24 08:34:56 2022] PUT /identity/v3/projects/d2f48107e8bb4875908f0d6dc2693154/groups/47495fe0b14a479bbf55b547abe0975d/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:34:56.433418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e83112-4edb-44f4-8630-b1e27d6b7e5b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:56.433814 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e83112-4edb-44f4-8630-b1e27d6b7e5b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:56.434022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e83112-4edb-44f4-8630-b1e27d6b7e5b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:56.435772 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 150/297] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:56.441167 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f13b3-e280-4ff0-acaa-289f467e5704 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:56.441467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f13b3-e280-4ff0-acaa-289f467e5704 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:56.441711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f13b3-e280-4ff0-acaa-289f467e5704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:56.465089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ee0f13b3-e280-4ff0-acaa-289f467e5704 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:56.466305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ee0f13b3-e280-4ff0-acaa-289f467e5704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:56.547818 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 148/298] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:34:57.006530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e83112-4edb-44f4-8630-b1e27d6b7e5b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:57.094538 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014c047-3785-46f4-86f4-c889384dadd3 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:57.095209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014c047-3785-46f4-86f4-c889384dadd3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:57.095209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014c047-3785-46f4-86f4-c889384dadd3 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:57.098448 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 149/299] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:57.109228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801c521-49b8-442e-aa13-86168f64ce78 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:57.109543 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801c521-49b8-442e-aa13-86168f64ce78 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:57.109792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801c521-49b8-442e-aa13-86168f64ce78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:57.136480 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a801c521-49b8-442e-aa13-86168f64ce78 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:57.137751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a801c521-49b8-442e-aa13-86168f64ce78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:57.158220 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e83112-4edb-44f4-8630-b1e27d6b7e5b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:57.201188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765305d-b0b8-424a-9095-90da126e0cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:57.202390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765305d-b0b8-424a-9095-90da126e0cf3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:57.202693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765305d-b0b8-424a-9095-90da126e0cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:57.202985 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765305d-b0b8-424a-9095-90da126e0cf3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:57.208283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:57.213607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:57.219689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765305d-b0b8-424a-9095-90da126e0cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:57.224419 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 150/300] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a765305d-b0b8-424a-9095-90da126e0cf3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:57.225816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:57.226822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:57.227830 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:57.228544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e0f54-1d2c-4e1f-b68b-35a4b452011a None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:57.228544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e0f54-1d2c-4e1f-b68b-35a4b452011a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:57.228544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e0f54-1d2c-4e1f-b68b-35a4b452011a None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:57.228746 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 151/301] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:57.231526 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 151/302] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:57.233783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765305d-b0b8-424a-9095-90da126e0cf3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:57.237255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13279784-793f-497e-87b5-1b5ad35d0fad None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:57.237533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13279784-793f-497e-87b5-1b5ad35d0fad None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:57.237801 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13279784-793f-497e-87b5-1b5ad35d0fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:57.259047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-13279784-793f-497e-87b5-1b5ad35d0fad None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:57.260426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-13279784-793f-497e-87b5-1b5ad35d0fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:57.345090 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 152/303] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:57.360512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765305d-b0b8-424a-9095-90da126e0cf3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:57.394668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda5475-c00e-4790-a875-e2c40d804ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:57.395618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda5475-c00e-4790-a875-e2c40d804ccc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:57.395906 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda5475-c00e-4790-a875-e2c40d804ccc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:57.396177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda5475-c00e-4790-a875-e2c40d804ccc None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:57.396839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda5475-c00e-4790-a875-e2c40d804ccc None admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:57.397138 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda5475-c00e-4790-a875-e2c40d804ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:57.398529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda5475-c00e-4790-a875-e2c40d804ccc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:57.407599 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 152/304] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:57.414827 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13279784-793f-497e-87b5-1b5ad35d0fad None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:57.520035 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13279784-793f-497e-87b5-1b5ad35d0fad None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:57.538093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:57.556216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb27fa-855a-4387-a894-e8e22e4d7d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:57.556928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb27fa-855a-4387-a894-e8e22e4d7d72 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:57.557207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb27fa-855a-4387-a894-e8e22e4d7d72 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:57.557476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb27fa-855a-4387-a894-e8e22e4d7d72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:57.589813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda5475-c00e-4790-a875-e2c40d804ccc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:57.639514 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb27fa-855a-4387-a894-e8e22e4d7d72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:57.640985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb27fa-855a-4387-a894-e8e22e4d7d72 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:57.666847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:57.759888 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 153/305] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:57.766747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb27fa-855a-4387-a894-e8e22e4d7d72 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:57.789872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda5475-c00e-4790-a875-e2c40d804ccc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:57.791146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:57.840208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4801f79-90ff-4140-b344-a960feddea52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:57.840859 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4801f79-90ff-4140-b344-a960feddea52 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:57.841132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4801f79-90ff-4140-b344-a960feddea52 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:57.841403 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4801f79-90ff-4140-b344-a960feddea52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:57.857517 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4801f79-90ff-4140-b344-a960feddea52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-d4801f79-90ff-4140-b344-a960feddea52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:57.862993 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:57.863860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 08:34:57.864649 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:57.869017 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 153/306] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:34:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:57.869926 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4801f79-90ff-4140-b344-a960feddea52 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:57.873537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb27fa-855a-4387-a894-e8e22e4d7d72 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:57.924861 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f1e07-abc3-49c5-8a97-3ef88d74352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:57.925511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f1e07-abc3-49c5-8a97-3ef88d74352b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:57.926101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f1e07-abc3-49c5-8a97-3ef88d74352b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:57.926383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f1e07-abc3-49c5-8a97-3ef88d74352b None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:57.929487 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f1e07-abc3-49c5-8a97-3ef88d74352b None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:57.931356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f1e07-abc3-49c5-8a97-3ef88d74352b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:57.933485 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 154/307] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:57 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:57.934307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f1e07-abc3-49c5-8a97-3ef88d74352b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:58.008517 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4801f79-90ff-4140-b344-a960feddea52 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:58.032607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:58.051312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbcf6b-889d-4afc-9bc6-fd495ae4e451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:58.052039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbcf6b-889d-4afc-9bc6-fd495ae4e451 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.052314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbcf6b-889d-4afc-9bc6-fd495ae4e451 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.052583 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbcf6b-889d-4afc-9bc6-fd495ae4e451 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.053885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:58.065363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbcf6b-889d-4afc-9bc6-fd495ae4e451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-38bbcf6b-889d-4afc-9bc6-fd495ae4e451 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:58.071126 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:58.073065 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:58.075025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:34:58.076362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f1e07-abc3-49c5-8a97-3ef88d74352b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:58.076447 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 154/308] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:34:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:58.076447 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbcf6b-889d-4afc-9bc6-fd495ae4e451 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:58.114660 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936d7c5-f667-4f8a-9ddd-5499eb0c446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:58.115294 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936d7c5-f667-4f8a-9ddd-5499eb0c446c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.115572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936d7c5-f667-4f8a-9ddd-5499eb0c446c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.115840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936d7c5-f667-4f8a-9ddd-5499eb0c446c None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.116503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936d7c5-f667-4f8a-9ddd-5499eb0c446c None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:58.116803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936d7c5-f667-4f8a-9ddd-5499eb0c446c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:58.118192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936d7c5-f667-4f8a-9ddd-5499eb0c446c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:58.130083 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 155/309] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:34:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:58.132024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936d7c5-f667-4f8a-9ddd-5499eb0c446c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:58.190768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbcf6b-889d-4afc-9bc6-fd495ae4e451 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:58.231888 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0db78-5fa6-45b0-974c-e12de9a18b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:58.232547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0db78-5fa6-45b0-974c-e12de9a18b15 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.232820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0db78-5fa6-45b0-974c-e12de9a18b15 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.233092 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0db78-5fa6-45b0-974c-e12de9a18b15 None admin] PATH_INFO: `/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.234223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0db78-5fa6-45b0-974c-e12de9a18b15 None admin] RBAC: Authorizing `identity:get_group(group_id=182e8ffbac2d4f89b7c20afe3b523cf0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:58.235754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0db78-5fa6-45b0-974c-e12de9a18b15 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:58.237428 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 155/310] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Oct 24 08:34:58 2022] GET /identity/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:58.238375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0db78-5fa6-45b0-974c-e12de9a18b15 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:58.275494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936d7c5-f667-4f8a-9ddd-5499eb0c446c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:58.296526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:58.322008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38bae1-a6ff-4ce6-94a2-58bc2f247afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:58.322547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38bae1-a6ff-4ce6-94a2-58bc2f247afc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.323012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38bae1-a6ff-4ce6-94a2-58bc2f247afc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.323285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38bae1-a6ff-4ce6-94a2-58bc2f247afc None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.324056 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38bae1-a6ff-4ce6-94a2-58bc2f247afc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:58.324409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38bae1-a6ff-4ce6-94a2-58bc2f247afc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:58.325823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38bae1-a6ff-4ce6-94a2-58bc2f247afc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:58.335007 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 156/311] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:34:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:58.335994 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38bae1-a6ff-4ce6-94a2-58bc2f247afc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:58.357001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:58.364987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0db78-5fa6-45b0-974c-e12de9a18b15 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:58.405868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28a886-db27-4f63-b753-b74845d10e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:58.406515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a886-db27-4f63-b753-b74845d10e14 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.406814 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a886-db27-4f63-b753-b74845d10e14 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.407086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a886-db27-4f63-b753-b74845d10e14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.468590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38bae1-a6ff-4ce6-94a2-58bc2f247afc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:58.488760 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28a886-db27-4f63-b753-b74845d10e14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:58.490542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28a886-db27-4f63-b753-b74845d10e14 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:58.507107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc8123-0922-4d72-ab7c-f55fefdcf1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:58.507717 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc8123-0922-4d72-ab7c-f55fefdcf1a3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.507993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc8123-0922-4d72-ab7c-f55fefdcf1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.508263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc8123-0922-4d72-ab7c-f55fefdcf1a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.508982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc8123-0922-4d72-ab7c-f55fefdcf1a3 None admin] RBAC: Adding query filter params (group.id=182e8ffbac2d4f89b7c20afe3b523cf0, role.id=290368e3589e425e99a9f4dbfe2b3823, scope.project.id=538dc9f6865b4b1a932b75e3be851bb7) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:58.509290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc8123-0922-4d72-ab7c-f55fefdcf1a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:58.510710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc8123-0922-4d72-ab7c-f55fefdcf1a3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:58.521544 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 157/312] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Oct 24 08:34:58 2022] GET /identity/v3/role_assignments?group.id=182e8ffbac2d4f89b7c20afe3b523cf0&scope.project.id=538dc9f6865b4b1a932b75e3be851bb7&role.id=290368e3589e425e99a9f4dbfe2b3823 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:58.522945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc8123-0922-4d72-ab7c-f55fefdcf1a3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:58.556615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:58.633981 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 156/313] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:58.635325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db080268-c798-4763-a90f-6a13fa085f5e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.635607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db080268-c798-4763-a90f-6a13fa085f5e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.635877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db080268-c798-4763-a90f-6a13fa085f5e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.637500 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:58.637689 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 157/314] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:58.644933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa990065-12e3-4cc4-aad0-33b3b1b450ed None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.644933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa990065-12e3-4cc4-aad0-33b3b1b450ed None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.644933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa990065-12e3-4cc4-aad0-33b3b1b450ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.670160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-aa990065-12e3-4cc4-aad0-33b3b1b450ed None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:58.670814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc8123-0922-4d72-ab7c-f55fefdcf1a3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:58.671637 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aa990065-12e3-4cc4-aad0-33b3b1b450ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:58.713375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8273bc8-d8e2-4a70-b9a2-e55d6cf11eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:58.714471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8273bc8-d8e2-4a70-b9a2-e55d6cf11eef None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.715004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8273bc8-d8e2-4a70-b9a2-e55d6cf11eef None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.716106 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8273bc8-d8e2-4a70-b9a2-e55d6cf11eef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.762822 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 158/315] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 24 08:34:58.763487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa990065-12e3-4cc4-aad0-33b3b1b450ed None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:58.812332 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8273bc8-d8e2-4a70-b9a2-e55d6cf11eef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:58.813791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8273bc8-d8e2-4a70-b9a2-e55d6cf11eef None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:58.861789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:58.885131 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa990065-12e3-4cc4-aad0-33b3b1b450ed None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:58.908788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:58.928000 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728ccf5-c775-47d3-9e97-d53404442c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:58.928647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728ccf5-c775-47d3-9e97-d53404442c99 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:58.928944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728ccf5-c775-47d3-9e97-d53404442c99 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:58.929865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728ccf5-c775-47d3-9e97-d53404442c99 None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:58.931060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728ccf5-c775-47d3-9e97-d53404442c99 None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:58.932984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728ccf5-c775-47d3-9e97-d53404442c99 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:58.935069 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 159/316] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:58 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:58.943834 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728ccf5-c775-47d3-9e97-d53404442c99 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:58.945306 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 158/317] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:34:58.953943 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8273bc8-d8e2-4a70-b9a2-e55d6cf11eef None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:59.060503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728ccf5-c775-47d3-9e97-d53404442c99 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:59.067980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8273bc8-d8e2-4a70-b9a2-e55d6cf11eef None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:59.095863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1d41a-445f-44cc-9889-8b1d335c2d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:59.096503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d41a-445f-44cc-9889-8b1d335c2d3d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.096776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d41a-445f-44cc-9889-8b1d335c2d3d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.097048 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d41a-445f-44cc-9889-8b1d335c2d3d None admin] PATH_INFO: `/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.098812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1d41a-445f-44cc-9889-8b1d335c2d3d None admin] RBAC: Authorizing `identity:get_group(group_id=182e8ffbac2d4f89b7c20afe3b523cf0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:59.100261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1d41a-445f-44cc-9889-8b1d335c2d3d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:59.102214 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 160/318] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Oct 24 08:34:58 2022] GET /identity/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:59.105476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe9b0f-8ab2-4ee8-b5af-07a0ab25229f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:59.106121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe9b0f-8ab2-4ee8-b5af-07a0ab25229f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.106396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe9b0f-8ab2-4ee8-b5af-07a0ab25229f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.106667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe9b0f-8ab2-4ee8-b5af-07a0ab25229f None admin] PATH_INFO: `/v3/projects/538dc9f6865b4b1a932b75e3be851bb7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.108140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1d41a-445f-44cc-9889-8b1d335c2d3d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:59.108251 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe9b0f-8ab2-4ee8-b5af-07a0ab25229f None admin] RBAC: Authorizing `identity:get_project(project_id=538dc9f6865b4b1a932b75e3be851bb7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:59.109683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe9b0f-8ab2-4ee8-b5af-07a0ab25229f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:59.112640 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 159/319] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:34:58 2022] GET /identity/v3/projects/538dc9f6865b4b1a932b75e3be851bb7 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:59.116872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe9b0f-8ab2-4ee8-b5af-07a0ab25229f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:59.127126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:59.165376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:59.241982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe9b0f-8ab2-4ee8-b5af-07a0ab25229f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:59.246282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1d41a-445f-44cc-9889-8b1d335c2d3d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:59.295904 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bea8e1-2133-4d32-8fc0-7bbcb780772e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:59.296660 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bea8e1-2133-4d32-8fc0-7bbcb780772e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.296944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bea8e1-2133-4d32-8fc0-7bbcb780772e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.297217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bea8e1-2133-4d32-8fc0-7bbcb780772e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.297949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bea8e1-2133-4d32-8fc0-7bbcb780772e None admin] RBAC: Adding query filter params (group.id=182e8ffbac2d4f89b7c20afe3b523cf0, role.id=290368e3589e425e99a9f4dbfe2b3823, scope.project.id=157d82dd257b4e0b929f245d80d41b62) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:59.298258 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bea8e1-2133-4d32-8fc0-7bbcb780772e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:59.299357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96bc61-25f9-4d8f-acc1-4f01e1c88156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:59.299736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bea8e1-2133-4d32-8fc0-7bbcb780772e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:59.299831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96bc61-25f9-4d8f-acc1-4f01e1c88156 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.300031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96bc61-25f9-4d8f-acc1-4f01e1c88156 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.300298 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96bc61-25f9-4d8f-acc1-4f01e1c88156 None admin] PATH_INFO: `/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.302586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96bc61-25f9-4d8f-acc1-4f01e1c88156 None admin] RBAC: Authorizing `identity:get_group(group_id=182e8ffbac2d4f89b7c20afe3b523cf0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:59.303776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96bc61-25f9-4d8f-acc1-4f01e1c88156 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:59.305420 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 160/320] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Oct 24 08:34:59 2022] GET /identity/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:59.306124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96bc61-25f9-4d8f-acc1-4f01e1c88156 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:59.308909 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 161/321] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Oct 24 08:34:59 2022] GET /identity/v3/role_assignments?group.id=182e8ffbac2d4f89b7c20afe3b523cf0&scope.project.id=157d82dd257b4e0b929f245d80d41b62&role.id=290368e3589e425e99a9f4dbfe2b3823 => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:59.311681 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bea8e1-2133-4d32-8fc0-7bbcb780772e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:59.427726 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:59.446344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96bc61-25f9-4d8f-acc1-4f01e1c88156 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:59.456786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bea8e1-2133-4d32-8fc0-7bbcb780772e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:59.460436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:59.500664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03895e31-63bd-45cc-8b89-dd1f9967129c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:59.500664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03895e31-63bd-45cc-8b89-dd1f9967129c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.500664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03895e31-63bd-45cc-8b89-dd1f9967129c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.500664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03895e31-63bd-45cc-8b89-dd1f9967129c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.511922 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bec46-7e0e-4835-89b8-4ccac49131c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:59.512547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bec46-7e0e-4835-89b8-4ccac49131c9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.512990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bec46-7e0e-4835-89b8-4ccac49131c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.513391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bec46-7e0e-4835-89b8-4ccac49131c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.513574 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03895e31-63bd-45cc-8b89-dd1f9967129c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:34:59.514128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bec46-7e0e-4835-89b8-4ccac49131c9 None admin] RBAC: Adding query filter params (group.id=182e8ffbac2d4f89b7c20afe3b523cf0, role.id=5cd206a783e44ac2829c7661b34bb6a7, scope.project.id=538dc9f6865b4b1a932b75e3be851bb7) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:59.514439 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bec46-7e0e-4835-89b8-4ccac49131c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:59.515950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bec46-7e0e-4835-89b8-4ccac49131c9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-03895e31-63bd-45cc-8b89-dd1f9967129c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:34:59.519478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:34:59.520398 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:34:59.521209 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 161/322] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:34:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:34:59.521982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199d99-6d75-4004-bb4a-758e595331cb None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.522050 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199d99-6d75-4004-bb4a-758e595331cb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.522284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199d99-6d75-4004-bb4a-758e595331cb None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.523748 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 162/323] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:34:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:34:59.525809 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 162/324] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Oct 24 08:34:59 2022] GET /identity/v3/role_assignments?group.id=182e8ffbac2d4f89b7c20afe3b523cf0&scope.project.id=538dc9f6865b4b1a932b75e3be851bb7&role.id=5cd206a783e44ac2829c7661b34bb6a7 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:34:59.529314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85bf2-634a-40d9-aa55-0520bf7321f9 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.529685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85bf2-634a-40d9-aa55-0520bf7321f9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.530017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85bf2-634a-40d9-aa55-0520bf7321f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.532924 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bec46-7e0e-4835-89b8-4ccac49131c9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:59.554095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-edc85bf2-634a-40d9-aa55-0520bf7321f9 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:34:59.555474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-edc85bf2-634a-40d9-aa55-0520bf7321f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:34:59.657661 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 163/325] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:34:59.658767 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bec46-7e0e-4835-89b8-4ccac49131c9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:59.702012 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d0ddb-885c-48fd-b78c-e9b31f7b0e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:59.702566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0ddb-885c-48fd-b78c-e9b31f7b0e34 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.702879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0ddb-885c-48fd-b78c-e9b31f7b0e34 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.703158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0ddb-885c-48fd-b78c-e9b31f7b0e34 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.703850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d0ddb-885c-48fd-b78c-e9b31f7b0e34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:34:59.704156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d0ddb-885c-48fd-b78c-e9b31f7b0e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:59.705565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d0ddb-885c-48fd-b78c-e9b31f7b0e34 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:34:59.715873 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 163/326] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:34:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:34:59.721523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85bf2-634a-40d9-aa55-0520bf7321f9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:59.837269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:59.840633 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85bf2-634a-40d9-aa55-0520bf7321f9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:34:59.876492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55de57d-b74d-47d2-990d-779e0bbb0481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:34:59.877099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55de57d-b74d-47d2-990d-779e0bbb0481 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:34:59.877373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55de57d-b74d-47d2-990d-779e0bbb0481 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:34:59.877639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55de57d-b74d-47d2-990d-779e0bbb0481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:34:59.939539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d0ddb-885c-48fd-b78c-e9b31f7b0e34 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:34:59.943445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:34:59.954654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55de57d-b74d-47d2-990d-779e0bbb0481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:34:59.958224 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55de57d-b74d-47d2-990d-779e0bbb0481 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:00.060686 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d0ddb-885c-48fd-b78c-e9b31f7b0e34 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:00.063020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:00.067614 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 164/327] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:35:00.074599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55de57d-b74d-47d2-990d-779e0bbb0481 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:00.095586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0410a40-bb0b-4498-bbae-eeebf6dca443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:00.096232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0410a40-bb0b-4498-bbae-eeebf6dca443 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:00.096513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0410a40-bb0b-4498-bbae-eeebf6dca443 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:00.096786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0410a40-bb0b-4498-bbae-eeebf6dca443 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:00.107032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0410a40-bb0b-4498-bbae-eeebf6dca443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e0410a40-bb0b-4498-bbae-eeebf6dca443 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:00.112321 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:00.113313 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:00.114157 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 164/328] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:34:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:00.119314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0410a40-bb0b-4498-bbae-eeebf6dca443 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:00.186797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55de57d-b74d-47d2-990d-779e0bbb0481 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:00.188709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:00.226061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fe803-2562-4943-aacd-f89397597e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:00.226628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fe803-2562-4943-aacd-f89397597e80 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:00.226985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fe803-2562-4943-aacd-f89397597e80 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:00.227270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fe803-2562-4943-aacd-f89397597e80 None admin] PATH_INFO: `/v3/projects/157d82dd257b4e0b929f245d80d41b62` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:00.228409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fe803-2562-4943-aacd-f89397597e80 None admin] RBAC: Authorizing `identity:get_project(project_id=157d82dd257b4e0b929f245d80d41b62)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:00.229866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fe803-2562-4943-aacd-f89397597e80 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:00.230140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0410a40-bb0b-4498-bbae-eeebf6dca443 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:00.231778 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 165/329] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:35:00 2022] GET /identity/v3/projects/157d82dd257b4e0b929f245d80d41b62 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:00.237336 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fe803-2562-4943-aacd-f89397597e80 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:00.273150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a60a5-ea69-4fde-b516-8153690ca110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:00.273770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a60a5-ea69-4fde-b516-8153690ca110 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:00.274045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a60a5-ea69-4fde-b516-8153690ca110 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:00.274413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a60a5-ea69-4fde-b516-8153690ca110 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:00.275692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a60a5-ea69-4fde-b516-8153690ca110 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:00.275692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a60a5-ea69-4fde-b516-8153690ca110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:00.276894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a60a5-ea69-4fde-b516-8153690ca110 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:00.290468 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 165/330] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:00.296275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a60a5-ea69-4fde-b516-8153690ca110 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:00.313878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:00.346440 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fe803-2562-4943-aacd-f89397597e80 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:00.378736 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c054e-3a53-48aa-8dce-9b0b4e9a57d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:00.379405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c054e-3a53-48aa-8dce-9b0b4e9a57d9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:00.379683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c054e-3a53-48aa-8dce-9b0b4e9a57d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:00.379953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c054e-3a53-48aa-8dce-9b0b4e9a57d9 None admin] PATH_INFO: `/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:00.381047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c054e-3a53-48aa-8dce-9b0b4e9a57d9 None admin] RBAC: Authorizing `identity:get_group(group_id=182e8ffbac2d4f89b7c20afe3b523cf0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:00.382481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c054e-3a53-48aa-8dce-9b0b4e9a57d9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:00.384090 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 166/331] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Oct 24 08:35:00 2022] GET /identity/v3/groups/182e8ffbac2d4f89b7c20afe3b523cf0 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:00.390010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c054e-3a53-48aa-8dce-9b0b4e9a57d9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:00.390437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a60a5-ea69-4fde-b516-8153690ca110 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:00.416159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:00.424993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0d625-7939-4749-b574-aa77ff6db0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:00.425609 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0d625-7939-4749-b574-aa77ff6db0cd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:00.425887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0d625-7939-4749-b574-aa77ff6db0cd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:00.426161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0d625-7939-4749-b574-aa77ff6db0cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:00.487477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c054e-3a53-48aa-8dce-9b0b4e9a57d9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:00.491360 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0d625-7939-4749-b574-aa77ff6db0cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:00.492830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0d625-7939-4749-b574-aa77ff6db0cd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:00.520802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3adce4-f4e9-4d2c-89af-b81a3a173515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:00.521662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3adce4-f4e9-4d2c-89af-b81a3a173515 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:00.521936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3adce4-f4e9-4d2c-89af-b81a3a173515 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:00.522204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3adce4-f4e9-4d2c-89af-b81a3a173515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:00.522957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3adce4-f4e9-4d2c-89af-b81a3a173515 None admin] RBAC: Adding query filter params (group.id=182e8ffbac2d4f89b7c20afe3b523cf0, role.id=5cd206a783e44ac2829c7661b34bb6a7, scope.project.id=157d82dd257b4e0b929f245d80d41b62) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:00.523261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3adce4-f4e9-4d2c-89af-b81a3a173515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:00.524657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3adce4-f4e9-4d2c-89af-b81a3a173515 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:00.527462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:00.531160 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 167/332] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Oct 24 08:35:00 2022] GET /identity/v3/role_assignments?group.id=182e8ffbac2d4f89b7c20afe3b523cf0&scope.project.id=157d82dd257b4e0b929f245d80d41b62&role.id=5cd206a783e44ac2829c7661b34bb6a7 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:00.593509 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 166/333] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 08:35:00.599687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3adce4-f4e9-4d2c-89af-b81a3a173515 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:00.711352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3adce4-f4e9-4d2c-89af-b81a3a173515 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:00.747525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760a91d-a583-4692-b157-d464fa806043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:00.748074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760a91d-a583-4692-b157-d464fa806043 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:00.748354 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760a91d-a583-4692-b157-d464fa806043 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:00.748626 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760a91d-a583-4692-b157-d464fa806043 None admin] PATH_INFO: `/v3/projects/d2f48107e8bb4875908f0d6dc2693154` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:00.749759 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760a91d-a583-4692-b157-d464fa806043 None admin] RBAC: Authorizing `identity:get_project(project_id=d2f48107e8bb4875908f0d6dc2693154)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:00.751222 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760a91d-a583-4692-b157-d464fa806043 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:00.752848 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 168/334] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:35:00 2022] GET /identity/v3/projects/d2f48107e8bb4875908f0d6dc2693154 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:00.758828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0d625-7939-4749-b574-aa77ff6db0cd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:00.785899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:00.857873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0d625-7939-4749-b574-aa77ff6db0cd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:00.895244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd32e2-532c-4fbc-af8e-dadbe2e4b4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:00.895886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd32e2-532c-4fbc-af8e-dadbe2e4b4bd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:00.896166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd32e2-532c-4fbc-af8e-dadbe2e4b4bd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:00.896432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd32e2-532c-4fbc-af8e-dadbe2e4b4bd None admin] PATH_INFO: `/v3/groups/47495fe0b14a479bbf55b547abe0975d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:00.897626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd32e2-532c-4fbc-af8e-dadbe2e4b4bd None admin] RBAC: Authorizing `identity:get_group(group_id=47495fe0b14a479bbf55b547abe0975d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:00.899112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd32e2-532c-4fbc-af8e-dadbe2e4b4bd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:00.900698 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 167/335] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Oct 24 08:35:00 2022] GET /identity/v3/groups/47495fe0b14a479bbf55b547abe0975d => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:00.907022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760a91d-a583-4692-b157-d464fa806043 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:00.974228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:01.008733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760a91d-a583-4692-b157-d464fa806043 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:01.048567 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ba6ea-d03a-4a15-bb26-5d819db3757f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:01.049156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba6ea-d03a-4a15-bb26-5d819db3757f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:01.049434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba6ea-d03a-4a15-bb26-5d819db3757f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:01.049701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba6ea-d03a-4a15-bb26-5d819db3757f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:01.050412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ba6ea-d03a-4a15-bb26-5d819db3757f None admin] RBAC: Adding query filter params (group.id=47495fe0b14a479bbf55b547abe0975d, role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.project.id=d2f48107e8bb4875908f0d6dc2693154) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:01.050753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ba6ea-d03a-4a15-bb26-5d819db3757f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:01.052163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ba6ea-d03a-4a15-bb26-5d819db3757f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:01.057990 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 169/336] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Oct 24 08:35:00 2022] GET /identity/v3/role_assignments?group.id=47495fe0b14a479bbf55b547abe0975d&scope.project.id=d2f48107e8bb4875908f0d6dc2693154&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:03.774012 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583585b4-1aef-43ac-914e-1b26389d975e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:03.775159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583585b4-1aef-43ac-914e-1b26389d975e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:03.775660 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583585b4-1aef-43ac-914e-1b26389d975e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:03.777551 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 168/337] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:03.783398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c4781-efcf-4ee4-b1a4-19a216b01d92 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:03.784206 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c4781-efcf-4ee4-b1a4-19a216b01d92 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:03.784682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c4781-efcf-4ee4-b1a4-19a216b01d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:03.812232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1b5c4781-efcf-4ee4-b1a4-19a216b01d92 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:03.813771 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1b5c4781-efcf-4ee4-b1a4-19a216b01d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:03.876392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719953c9-5f79-4311-b173-6a8ba30bfe94 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:03.877033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719953c9-5f79-4311-b173-6a8ba30bfe94 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:03.877528 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719953c9-5f79-4311-b173-6a8ba30bfe94 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:03.879108 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 169/338] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:35:03.884920 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a561ef-b5ef-489a-b693-5a1390bec67e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:03.885431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a561ef-b5ef-489a-b693-5a1390bec67e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:03.885914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a561ef-b5ef-489a-b693-5a1390bec67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:03.913058 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c5a561ef-b5ef-489a-b693-5a1390bec67e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:03.914609 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c5a561ef-b5ef-489a-b693-5a1390bec67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:03.934694 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 170/339] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:35:04.015068 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 170/340] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:35:04.106847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f0f63-43c4-423a-9131-2aaf9b053e09 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:04.107477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f0f63-43c4-423a-9131-2aaf9b053e09 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:04.107952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f0f63-43c4-423a-9131-2aaf9b053e09 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:04.109764 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 171/341] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:04.115498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dc2a0-c01f-4254-8923-1fbd83fac04b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:04.115786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dc2a0-c01f-4254-8923-1fbd83fac04b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:04.116061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dc2a0-c01f-4254-8923-1fbd83fac04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:04.139692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f93dc2a0-c01f-4254-8923-1fbd83fac04b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:04.140878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f93dc2a0-c01f-4254-8923-1fbd83fac04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:04.162736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:04.229153 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 171/342] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:04.252559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f0f63-43c4-423a-9131-2aaf9b053e09 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:04.355243 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:04.403820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f0f63-43c4-423a-9131-2aaf9b053e09 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:04.449776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8d090-f01d-4e88-b5ce-5db7f16f9c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:04.450445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8d090-f01d-4e88-b5ce-5db7f16f9c4d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:04.450731 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8d090-f01d-4e88-b5ce-5db7f16f9c4d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:04.451004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8d090-f01d-4e88-b5ce-5db7f16f9c4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:04.465406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8d090-f01d-4e88-b5ce-5db7f16f9c4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:04.466890 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8d090-f01d-4e88-b5ce-5db7f16f9c4d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d6a8d090-f01d-4e88-b5ce-5db7f16f9c4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:04.491341 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:04.492380 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:04.493886 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:04.493886 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:04.493886 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:04.493886 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 172/343] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:04.496874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dc2a0-c01f-4254-8923-1fbd83fac04b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:04.498418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8d090-f01d-4e88-b5ce-5db7f16f9c4d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:04.607502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8d090-f01d-4e88-b5ce-5db7f16f9c4d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:04.621554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:04.626871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dc2a0-c01f-4254-8923-1fbd83fac04b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:04.645427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab29d5-84b1-4554-8bb1-b0f4a20e223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:04.646033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab29d5-84b1-4554-8bb1-b0f4a20e223a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:04.646308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab29d5-84b1-4554-8bb1-b0f4a20e223a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:04.646593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab29d5-84b1-4554-8bb1-b0f4a20e223a None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:04.647431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab29d5-84b1-4554-8bb1-b0f4a20e223a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:04.648005 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab29d5-84b1-4554-8bb1-b0f4a20e223a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:04.649111 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab29d5-84b1-4554-8bb1-b0f4a20e223a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:04.662020 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 173/344] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:04.663043 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63917f6c-53e9-4b61-9f53-54511f2b587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:04.663695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63917f6c-53e9-4b61-9f53-54511f2b587c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:04.663970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63917f6c-53e9-4b61-9f53-54511f2b587c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:04.664241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63917f6c-53e9-4b61-9f53-54511f2b587c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:04.667057 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab29d5-84b1-4554-8bb1-b0f4a20e223a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:04.675159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63917f6c-53e9-4b61-9f53-54511f2b587c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:04.676613 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63917f6c-53e9-4b61-9f53-54511f2b587c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-63917f6c-53e9-4b61-9f53-54511f2b587c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:04.692066 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:04.698415 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:04.699432 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:04.699432 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:04.699432 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:04.699432 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 172/345] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:04.699432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd053b5-c373-4cf0-a917-b6a2bc85f1a7 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:04.699432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd053b5-c373-4cf0-a917-b6a2bc85f1a7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:04.699432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd053b5-c373-4cf0-a917-b6a2bc85f1a7 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:04.699432 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 173/346] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:04.699432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd053b5-c373-4cf0-a917-b6a2bc85f1a7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:04.707781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:04.780245 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab29d5-84b1-4554-8bb1-b0f4a20e223a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:04.803599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd053b5-c373-4cf0-a917-b6a2bc85f1a7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:04.818633 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a764e4-ad8e-46b7-b80c-01fe8311bdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:04.819305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a764e4-ad8e-46b7-b80c-01fe8311bdf2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:04.819578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a764e4-ad8e-46b7-b80c-01fe8311bdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:04.819846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a764e4-ad8e-46b7-b80c-01fe8311bdf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:04.830218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a764e4-ad8e-46b7-b80c-01fe8311bdf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:04.831673 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a764e4-ad8e-46b7-b80c-01fe8311bdf2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:04.837711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9dbca-81c0-45d5-a652-d1f1041d2a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:04.838327 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9dbca-81c0-45d5-a652-d1f1041d2a21 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:04.838607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9dbca-81c0-45d5-a652-d1f1041d2a21 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:04.838925 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9dbca-81c0-45d5-a652-d1f1041d2a21 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:04.839617 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9dbca-81c0-45d5-a652-d1f1041d2a21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:04.841129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9dbca-81c0-45d5-a652-d1f1041d2a21 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-69a764e4-ad8e-46b7-b80c-01fe8311bdf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:04.846656 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:04.847673 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:04.848548 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:04.848548 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:04.848548 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:04.848548 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 174/347] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:04.849140 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a764e4-ad8e-46b7-b80c-01fe8311bdf2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:04.877136 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 174/348] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:35:04 2022] POST /identity/v3/users => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:04.877136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b267fc2-2792-4b15-8fb2-db250325f4ad None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:04.877136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b267fc2-2792-4b15-8fb2-db250325f4ad None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:04.877136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b267fc2-2792-4b15-8fb2-db250325f4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:04.889030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:04.892800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8b267fc2-2792-4b15-8fb2-db250325f4ad None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:04.894045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8b267fc2-2792-4b15-8fb2-db250325f4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:04.963062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:04.978324 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 175/349] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:04.979333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b267fc2-2792-4b15-8fb2-db250325f4ad None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:05.020755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a764e4-ad8e-46b7-b80c-01fe8311bdf2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:05.083776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce806f69-44d1-4f00-9abe-cc8ac7923b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:05.084832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce806f69-44d1-4f00-9abe-cc8ac7923b78 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:05.085309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce806f69-44d1-4f00-9abe-cc8ac7923b78 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:05.085874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce806f69-44d1-4f00-9abe-cc8ac7923b78 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:05.086770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce806f69-44d1-4f00-9abe-cc8ac7923b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:05.088090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce806f69-44d1-4f00-9abe-cc8ac7923b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:05.089956 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce806f69-44d1-4f00-9abe-cc8ac7923b78 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:05.112198 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 175/350] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:05.114675 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce806f69-44d1-4f00-9abe-cc8ac7923b78 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:05.141671 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b267fc2-2792-4b15-8fb2-db250325f4ad None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:05.185349 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783de38e-0194-433f-9908-203a091bd8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:05.185968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783de38e-0194-433f-9908-203a091bd8c8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:05.186245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783de38e-0194-433f-9908-203a091bd8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:05.186513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783de38e-0194-433f-9908-203a091bd8c8 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:05.187242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783de38e-0194-433f-9908-203a091bd8c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:05.187541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783de38e-0194-433f-9908-203a091bd8c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:05.189108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783de38e-0194-433f-9908-203a091bd8c8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:05.201201 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 176/351] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:05.202793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78681c0-b494-4ce9-9e07-f2e95252c4c8 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:05.203294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78681c0-b494-4ce9-9e07-f2e95252c4c8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:05.203792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78681c0-b494-4ce9-9e07-f2e95252c4c8 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:05.205434 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 177/352] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:05.206654 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78681c0-b494-4ce9-9e07-f2e95252c4c8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:05.233125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:05.251345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce806f69-44d1-4f00-9abe-cc8ac7923b78 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:05.267680 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:05.290535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca9b6f-6429-4d63-b296-4f593c7a3ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:05.291279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca9b6f-6429-4d63-b296-4f593c7a3ce3 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:05.291673 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca9b6f-6429-4d63-b296-4f593c7a3ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:05.292041 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca9b6f-6429-4d63-b296-4f593c7a3ce3 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:05.292967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca9b6f-6429-4d63-b296-4f593c7a3ce3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:05.294643 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca9b6f-6429-4d63-b296-4f593c7a3ce3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:05.317279 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78681c0-b494-4ce9-9e07-f2e95252c4c8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:05.326196 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 176/353] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:35:05 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:05.326196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd680b-b382-40f5-b797-012bc2ce55f0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:05.326196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd680b-b382-40f5-b797-012bc2ce55f0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:05.326196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd680b-b382-40f5-b797-012bc2ce55f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:05.343568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e0dd680b-b382-40f5-b797-012bc2ce55f0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:05.344833 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e0dd680b-b382-40f5-b797-012bc2ce55f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:05.366296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19bd1d-cbf9-48eb-9ac4-d6f57414b713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:05.367070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19bd1d-cbf9-48eb-9ac4-d6f57414b713 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:05.367419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19bd1d-cbf9-48eb-9ac4-d6f57414b713 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:05.367760 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19bd1d-cbf9-48eb-9ac4-d6f57414b713 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:05.368529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19bd1d-cbf9-48eb-9ac4-d6f57414b713 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:05.370070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19bd1d-cbf9-48eb-9ac4-d6f57414b713 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:05.399408 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 178/354] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:35:05 2022] POST /identity/v3/users => generated 248 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:35:05.400775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19bd1d-cbf9-48eb-9ac4-d6f57414b713 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:05.451548 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 177/355] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:05.583072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19bd1d-cbf9-48eb-9ac4-d6f57414b713 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:05.593367 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:05.623486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09bff0-8ef1-43e2-9b94-f5e31e0239df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:05.624144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09bff0-8ef1-43e2-9b94-f5e31e0239df None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:05.624426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09bff0-8ef1-43e2-9b94-f5e31e0239df None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:05.624702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09bff0-8ef1-43e2-9b94-f5e31e0239df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:05.636248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09bff0-8ef1-43e2-9b94-f5e31e0239df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:05.637785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09bff0-8ef1-43e2-9b94-f5e31e0239df None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-bd09bff0-8ef1-43e2-9b94-f5e31e0239df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:05.652637 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:05.653580 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:05.654460 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:05.654460 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:05.654460 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:05.654460 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 179/356] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:05.661181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd680b-b382-40f5-b797-012bc2ce55f0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:05.742931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:05.759095 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09bff0-8ef1-43e2-9b94-f5e31e0239df None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:05.776615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd680b-b382-40f5-b797-012bc2ce55f0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:05.816480 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091e802-3b65-4247-b80c-7aa472c31c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:05.817658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091e802-3b65-4247-b80c-7aa472c31c70 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:05.818276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091e802-3b65-4247-b80c-7aa472c31c70 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:05.818747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091e802-3b65-4247-b80c-7aa472c31c70 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:05.820148 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091e802-3b65-4247-b80c-7aa472c31c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:05.820688 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091e802-3b65-4247-b80c-7aa472c31c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:05.822395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091e802-3b65-4247-b80c-7aa472c31c70 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:05.837490 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 178/357] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:05.844368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091e802-3b65-4247-b80c-7aa472c31c70 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:05.887685 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09bff0-8ef1-43e2-9b94-f5e31e0239df None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:05.954660 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:05.955821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:05.957316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:05.957787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:05.958858 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:05.961213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:05.978547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.sql.core [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 08:35:05.978547 dv-cinder-master-3-smb devstack@keystone.service[87873]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 24 08:35:05.978547 dv-cinder-master-3-smb devstack@keystone.service[87873]: [parameters: {'id': '5e82fb3839de4f6ab28e8b89cc1aad84', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 24 08:35:05.978547 dv-cinder-master-3-smb devstack@keystone.service[87873]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87873) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Oct 24 08:35:06.005490 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return ref.to_dict() Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:35:06.006411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application yield resource Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise self.value Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application session.commit() Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 08:35:06.007375 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.flush() Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application compat.raise_( Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 08:35:06.008210 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:35:06.009109 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:35:06.009914 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [parameters: {'id': '5e82fb3839de4f6ab28e8b89cc1aad84', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:06.011822 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 24 08:35:06.013734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:06.021946 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 180/358] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Oct 24 08:35:05 2022] POST /identity/v3/roles => generated 144 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:06.028337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:06.056772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091e802-3b65-4247-b80c-7aa472c31c70 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:06.085156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:06.096876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:06.103097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e01c29-c986-495e-9a11-55467ab18909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:06.104324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e01c29-c986-495e-9a11-55467ab18909 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:06.104810 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e01c29-c986-495e-9a11-55467ab18909 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:06.105469 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e01c29-c986-495e-9a11-55467ab18909 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:06.107116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e01c29-c986-495e-9a11-55467ab18909 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:06.109317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e01c29-c986-495e-9a11-55467ab18909 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:06.132635 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 179/359] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:35:05 2022] POST /identity/v3/users => generated 249 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:06.243776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933db885-4c34-4c4d-a39c-92ef58912a07 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:06.305308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eefc1b-f908-4e06-819c-449d73a425bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:06.305986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eefc1b-f908-4e06-819c-449d73a425bc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:06.306266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eefc1b-f908-4e06-819c-449d73a425bc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:06.306540 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eefc1b-f908-4e06-819c-449d73a425bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:06.327767 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eefc1b-f908-4e06-819c-449d73a425bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-89eefc1b-f908-4e06-819c-449d73a425bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:06.333256 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:06.334202 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:06.335041 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 181/360] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:06.341960 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e01c29-c986-495e-9a11-55467ab18909 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:06.460931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e01c29-c986-495e-9a11-55467ab18909 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:06.495668 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306e8d5-04f5-4dc3-acf3-ea280d77f4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:06.496735 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e8d5-04f5-4dc3-acf3-ea280d77f4af None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:06.497248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e8d5-04f5-4dc3-acf3-ea280d77f4af None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:06.497716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306e8d5-04f5-4dc3-acf3-ea280d77f4af None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:06.498564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306e8d5-04f5-4dc3-acf3-ea280d77f4af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:06.499080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306e8d5-04f5-4dc3-acf3-ea280d77f4af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:06.501062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306e8d5-04f5-4dc3-acf3-ea280d77f4af None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:06.509775 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 180/361] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:07.366790 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a1911-7523-42bc-9fb3-3f02307092a8 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:07.367310 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a1911-7523-42bc-9fb3-3f02307092a8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:07.367411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a1911-7523-42bc-9fb3-3f02307092a8 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:07.368962 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 182/362] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:07.374562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0004a-784d-409b-bdec-c975d7dfb4e3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:07.375388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0004a-784d-409b-bdec-c975d7dfb4e3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:07.375850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0004a-784d-409b-bdec-c975d7dfb4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:07.395972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f0a0004a-784d-409b-bdec-c975d7dfb4e3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:07.398001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f0a0004a-784d-409b-bdec-c975d7dfb4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:07.453023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:07.482502 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 181/363] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:07.920265 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a319211-cafe-4273-9df6-8736056a234c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:07.920647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a319211-cafe-4273-9df6-8736056a234c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:07.920839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a319211-cafe-4273-9df6-8736056a234c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:07.922463 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 183/364] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:07.927392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ad21b-5499-4380-aa89-f2fdba598ca1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:07.927686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ad21b-5499-4380-aa89-f2fdba598ca1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:07.927966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ad21b-5499-4380-aa89-f2fdba598ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:07.946360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-432ad21b-5499-4380-aa89-f2fdba598ca1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:07.947602 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-432ad21b-5499-4380-aa89-f2fdba598ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:07.986591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a319211-cafe-4273-9df6-8736056a234c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:08.032920 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 182/365] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:35:08.040205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96a974-3d7b-42b6-9115-c89fb96fa2b6 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.040493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96a974-3d7b-42b6-9115-c89fb96fa2b6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.040773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96a974-3d7b-42b6-9115-c89fb96fa2b6 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.042340 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 183/366] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:08.047612 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4d39a-ab7b-4818-b2fe-e76214f9cafd None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.047890 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4d39a-ab7b-4818-b2fe-e76214f9cafd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.048167 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4d39a-ab7b-4818-b2fe-e76214f9cafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.083889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c4d4d39a-ab7b-4818-b2fe-e76214f9cafd None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:08.085089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c4d4d39a-ab7b-4818-b2fe-e76214f9cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:08.119076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:08.143379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a319211-cafe-4273-9df6-8736056a234c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:08.209720 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ef5bc-223d-4c08-b8fa-6cf818efb2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:08.210292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ef5bc-223d-4c08-b8fa-6cf818efb2b5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.210573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ef5bc-223d-4c08-b8fa-6cf818efb2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.210874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ef5bc-223d-4c08-b8fa-6cf818efb2b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.219343 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 184/367] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:35:08.225548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ef5bc-223d-4c08-b8fa-6cf818efb2b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-658ef5bc-223d-4c08-b8fa-6cf818efb2b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:08.230435 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:08.231412 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:08.232225 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 184/368] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:08.244055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4d39a-ab7b-4818-b2fe-e76214f9cafd None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:08.327137 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:08.377487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ef5bc-223d-4c08-b8fa-6cf818efb2b5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:08.427893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4d39a-ab7b-4818-b2fe-e76214f9cafd None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:08.493980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2715a-f91e-4205-8a80-04149d3890d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:08.495152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2715a-f91e-4205-8a80-04149d3890d9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.495630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2715a-f91e-4205-8a80-04149d3890d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.496865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2715a-f91e-4205-8a80-04149d3890d9 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.498035 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2715a-f91e-4205-8a80-04149d3890d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:08.498537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2715a-f91e-4205-8a80-04149d3890d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:08.500356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2715a-f91e-4205-8a80-04149d3890d9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:08.509378 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 185/369] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:08.510210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2715a-f91e-4205-8a80-04149d3890d9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:08.538462 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ef5bc-223d-4c08-b8fa-6cf818efb2b5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:08.574172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef2b23-325f-40c4-8294-39815dd79128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:08.575365 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef2b23-325f-40c4-8294-39815dd79128 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.575840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef2b23-325f-40c4-8294-39815dd79128 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.576320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef2b23-325f-40c4-8294-39815dd79128 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.587038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef2b23-325f-40c4-8294-39815dd79128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-6cef2b23-325f-40c4-8294-39815dd79128 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:08.592677 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:08.593579 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:08.594398 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:08.596973 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 185/370] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:35:08.598143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef2b23-325f-40c4-8294-39815dd79128 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:08.602280 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:08.616523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2715a-f91e-4205-8a80-04149d3890d9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:08.618073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:08.653171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df5852-d3e9-4917-8835-87f46106f3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:08.653788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df5852-d3e9-4917-8835-87f46106f3c3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.654069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df5852-d3e9-4917-8835-87f46106f3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.654394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df5852-d3e9-4917-8835-87f46106f3c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.665404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df5852-d3e9-4917-8835-87f46106f3c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d6df5852-d3e9-4917-8835-87f46106f3c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:08.670867 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:08.671771 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:08.672570 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 186/371] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:08.673293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df5852-d3e9-4917-8835-87f46106f3c3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:08.719631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef2b23-325f-40c4-8294-39815dd79128 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:08.756156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eed878-4fd1-4718-9828-8d4d67ab4704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:08.757135 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eed878-4fd1-4718-9828-8d4d67ab4704 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.757624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eed878-4fd1-4718-9828-8d4d67ab4704 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.758083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eed878-4fd1-4718-9828-8d4d67ab4704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.771552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eed878-4fd1-4718-9828-8d4d67ab4704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:08.773818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eed878-4fd1-4718-9828-8d4d67ab4704 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:08.780132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df5852-d3e9-4917-8835-87f46106f3c3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-c1eed878-4fd1-4718-9828-8d4d67ab4704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:08.793818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:08.794770 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:08.795635 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:08.795635 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:08.795635 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:08.796515 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 186/372] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:08.797058 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eed878-4fd1-4718-9828-8d4d67ab4704 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:08.818812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0f076-0c9e-401d-a361-35e327067977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:08.819423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0f076-0c9e-401d-a361-35e327067977 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.819706 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0f076-0c9e-401d-a361-35e327067977 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.819981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0f076-0c9e-401d-a361-35e327067977 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.820703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0f076-0c9e-401d-a361-35e327067977 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:08.821006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0f076-0c9e-401d-a361-35e327067977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:08.822409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0f076-0c9e-401d-a361-35e327067977 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:08.834635 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 187/373] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:08.834635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0f076-0c9e-401d-a361-35e327067977 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:08.859317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:08.863656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:08.909429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eed878-4fd1-4718-9828-8d4d67ab4704 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:08.940775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0f076-0c9e-401d-a361-35e327067977 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:08.943154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a86a1b-715f-4dc4-af19-5e81437b2f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:08.943850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a86a1b-715f-4dc4-af19-5e81437b2f37 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.944137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a86a1b-715f-4dc4-af19-5e81437b2f37 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.944409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a86a1b-715f-4dc4-af19-5e81437b2f37 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.945077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a86a1b-715f-4dc4-af19-5e81437b2f37 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:08.945379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a86a1b-715f-4dc4-af19-5e81437b2f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:08.946767 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a86a1b-715f-4dc4-af19-5e81437b2f37 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:08.954848 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 187/374] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:08.956054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856d9e8-2f74-4b6d-aae4-cf599b7b2c61 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.956347 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856d9e8-2f74-4b6d-aae4-cf599b7b2c61 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.956614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856d9e8-2f74-4b6d-aae4-cf599b7b2c61 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.962199 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 188/375] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:08.962199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856d9e8-2f74-4b6d-aae4-cf599b7b2c61 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:08.983255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3295d0-e32d-4cd2-a460-30ce47f77d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:08.983869 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3295d0-e32d-4cd2-a460-30ce47f77d15 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:08.984149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3295d0-e32d-4cd2-a460-30ce47f77d15 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:08.984421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3295d0-e32d-4cd2-a460-30ce47f77d15 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:08.985061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3295d0-e32d-4cd2-a460-30ce47f77d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:08.985362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3295d0-e32d-4cd2-a460-30ce47f77d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:08.986729 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3295d0-e32d-4cd2-a460-30ce47f77d15 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.002792 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 188/376] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:09.002792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3295d0-e32d-4cd2-a460-30ce47f77d15 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.080732 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856d9e8-2f74-4b6d-aae4-cf599b7b2c61 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.083796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:09.099921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:09.100089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3295d0-e32d-4cd2-a460-30ce47f77d15 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.117469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e5a16-ea03-4684-9bad-c3b2e4b5bf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.118104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e5a16-ea03-4684-9bad-c3b2e4b5bf20 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.118379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e5a16-ea03-4684-9bad-c3b2e4b5bf20 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.118824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e5a16-ea03-4684-9bad-c3b2e4b5bf20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.129502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e5a16-ea03-4684-9bad-c3b2e4b5bf20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:09.131109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e5a16-ea03-4684-9bad-c3b2e4b5bf20 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.136209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4dc2-d189-42f2-8b90-116ad609d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.136917 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4dc2-d189-42f2-8b90-116ad609d627 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.137607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4dc2-d189-42f2-8b90-116ad609d627 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.137889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d4dc2-d189-42f2-8b90-116ad609d627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-837e5a16-ea03-4684-9bad-c3b2e4b5bf20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:09.145190 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:09.146220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:09.147414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:09.147414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:09.147414 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:09.147414 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 189/377] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:09.147414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30092e2-ddfb-4c1f-aad6-d219532ea634 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.147691 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30092e2-ddfb-4c1f-aad6-d219532ea634 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.147925 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30092e2-ddfb-4c1f-aad6-d219532ea634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.154330 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d4dc2-d189-42f2-8b90-116ad609d627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:09.155828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d4dc2-d189-42f2-8b90-116ad609d627 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.166277 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b30092e2-ddfb-4c1f-aad6-d219532ea634 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:09.167518 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b30092e2-ddfb-4c1f-aad6-d219532ea634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f67d4dc2-d189-42f2-8b90-116ad609d627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:09.175619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:09.176545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:09.177383 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:09.177383 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:09.177383 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:09.177383 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 189/378] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:09.177606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4dc2-d189-42f2-8b90-116ad609d627 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.242790 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 190/379] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:09.244003 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30092e2-ddfb-4c1f-aad6-d219532ea634 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.297459 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d4dc2-d189-42f2-8b90-116ad609d627 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.335096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab1e59-33fe-4bf3-9f4d-d86d4bb5f45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.335963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab1e59-33fe-4bf3-9f4d-d86d4bb5f45b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.336383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab1e59-33fe-4bf3-9f4d-d86d4bb5f45b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.336878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab1e59-33fe-4bf3-9f4d-d86d4bb5f45b None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.340598 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:09.346662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab1e59-33fe-4bf3-9f4d-d86d4bb5f45b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:09.348747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab1e59-33fe-4bf3-9f4d-d86d4bb5f45b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.361031 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e3ab1e59-33fe-4bf3-9f4d-d86d4bb5f45b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:09.363993 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:09.365047 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:35:09.365965 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:09.366485 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30092e2-ddfb-4c1f-aad6-d219532ea634 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.368273 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 190/380] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:09.369360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab1e59-33fe-4bf3-9f4d-d86d4bb5f45b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.420903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd7063-1c59-4dab-befd-6c3c8f3f474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.421585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd7063-1c59-4dab-befd-6c3c8f3f474c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.421803 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd7063-1c59-4dab-befd-6c3c8f3f474c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.422078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd7063-1c59-4dab-befd-6c3c8f3f474c None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.422749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd7063-1c59-4dab-befd-6c3c8f3f474c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:09.423071 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd7063-1c59-4dab-befd-6c3c8f3f474c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:09.424915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd7063-1c59-4dab-befd-6c3c8f3f474c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.444677 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 191/381] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:09.444677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd7063-1c59-4dab-befd-6c3c8f3f474c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.496681 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab1e59-33fe-4bf3-9f4d-d86d4bb5f45b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.533331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66878629-206e-4644-841d-5bc0c3f19d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.534534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878629-206e-4644-841d-5bc0c3f19d51 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.535521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878629-206e-4644-841d-5bc0c3f19d51 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.536123 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878629-206e-4644-841d-5bc0c3f19d51 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.537401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66878629-206e-4644-841d-5bc0c3f19d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:09.538010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66878629-206e-4644-841d-5bc0c3f19d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:09.539743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66878629-206e-4644-841d-5bc0c3f19d51 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.552615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd7063-1c59-4dab-befd-6c3c8f3f474c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.553677 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 191/382] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:09.554488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66878629-206e-4644-841d-5bc0c3f19d51 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.589879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ee3a1-0eb1-4c44-b740-4cc20bc673f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.590403 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ee3a1-0eb1-4c44-b740-4cc20bc673f0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.590671 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ee3a1-0eb1-4c44-b740-4cc20bc673f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.590978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ee3a1-0eb1-4c44-b740-4cc20bc673f0 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.591692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ee3a1-0eb1-4c44-b740-4cc20bc673f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:09.591989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ee3a1-0eb1-4c44-b740-4cc20bc673f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:09.593402 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ee3a1-0eb1-4c44-b740-4cc20bc673f0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.611621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:09.629034 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 192/383] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:09.630274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339330-dc07-42bc-83c7-2552de77ae15 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.630550 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339330-dc07-42bc-83c7-2552de77ae15 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.630848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339330-dc07-42bc-83c7-2552de77ae15 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.632201 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 193/384] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:09.634798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2339330-dc07-42bc-83c7-2552de77ae15 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.653565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:09.656498 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66878629-206e-4644-841d-5bc0c3f19d51 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.689814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8b4d3-7212-4590-8628-19b36131170c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.690445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8b4d3-7212-4590-8628-19b36131170c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.690731 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8b4d3-7212-4590-8628-19b36131170c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.691027 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8b4d3-7212-4590-8628-19b36131170c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.697860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8b4d3-7212-4590-8628-19b36131170c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:09.699425 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8b4d3-7212-4590-8628-19b36131170c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-92f8b4d3-7212-4590-8628-19b36131170c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:09.710735 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:09.711648 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 192/385] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:09.712991 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8b4d3-7212-4590-8628-19b36131170c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.738164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2339330-dc07-42bc-83c7-2552de77ae15 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.770787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cf1f2-7a3f-42d8-bfc0-163f9df9dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.771618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cf1f2-7a3f-42d8-bfc0-163f9df9dd51 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.771893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cf1f2-7a3f-42d8-bfc0-163f9df9dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.772161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cf1f2-7a3f-42d8-bfc0-163f9df9dd51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.781967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cf1f2-7a3f-42d8-bfc0-163f9df9dd51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-5b0cf1f2-7a3f-42d8-bfc0-163f9df9dd51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:09.787220 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:09.788132 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:09.788945 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 194/386] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:09.789612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cf1f2-7a3f-42d8-bfc0-163f9df9dd51 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.814952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8b4d3-7212-4590-8628-19b36131170c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.831477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:09.849537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6c492-7286-4d13-be83-35b8b8f3ffd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.850091 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6c492-7286-4d13-be83-35b8b8f3ffd4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.850351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6c492-7286-4d13-be83-35b8b8f3ffd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.850618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6c492-7286-4d13-be83-35b8b8f3ffd4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.858186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6c492-7286-4d13-be83-35b8b8f3ffd4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:09.859679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6c492-7286-4d13-be83-35b8b8f3ffd4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.866877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-52c6c492-7286-4d13-be83-35b8b8f3ffd4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:09.870951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:09.871910 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:09.872795 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 193/387] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:09.873353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43327583-6764-4b4a-9df3-4c45124d0d67 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.873417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43327583-6764-4b4a-9df3-4c45124d0d67 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.873655 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43327583-6764-4b4a-9df3-4c45124d0d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.892693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cf1f2-7a3f-42d8-bfc0-163f9df9dd51 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:09.895174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-43327583-6764-4b4a-9df3-4c45124d0d67 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:09.896406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-43327583-6764-4b4a-9df3-4c45124d0d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:09.928571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e04a0f-9837-49d7-91e9-fb256ce4568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:09.929207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e04a0f-9837-49d7-91e9-fb256ce4568d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:09.929484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e04a0f-9837-49d7-91e9-fb256ce4568d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:09.929759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e04a0f-9837-49d7-91e9-fb256ce4568d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:09.939643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e04a0f-9837-49d7-91e9-fb256ce4568d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:09.941067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e04a0f-9837-49d7-91e9-fb256ce4568d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e7e04a0f-9837-49d7-91e9-fb256ce4568d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:09.955399 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:09.956488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:09.957419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:09.957419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:09.957419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:09.957419 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 195/388] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:09.957419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e04a0f-9837-49d7-91e9-fb256ce4568d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:09.966686 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 194/389] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:09.967525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43327583-6764-4b4a-9df3-4c45124d0d67 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.070957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43327583-6764-4b4a-9df3-4c45124d0d67 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.071403 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e04a0f-9837-49d7-91e9-fb256ce4568d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.079288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:10.099632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:10.106299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2995e-b19a-4049-ad6b-568101e0f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.106934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2995e-b19a-4049-ad6b-568101e0f338 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.107209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2995e-b19a-4049-ad6b-568101e0f338 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.107477 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2995e-b19a-4049-ad6b-568101e0f338 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.108193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2995e-b19a-4049-ad6b-568101e0f338 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:10.108500 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2995e-b19a-4049-ad6b-568101e0f338 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.109279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27139097-f1b5-4372-86a5-22eaa41bcdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.109890 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27139097-f1b5-4372-86a5-22eaa41bcdc9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.109982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2995e-b19a-4049-ad6b-568101e0f338 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.110170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27139097-f1b5-4372-86a5-22eaa41bcdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.110445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27139097-f1b5-4372-86a5-22eaa41bcdc9 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.111155 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27139097-f1b5-4372-86a5-22eaa41bcdc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:10.111537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27139097-f1b5-4372-86a5-22eaa41bcdc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.113345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27139097-f1b5-4372-86a5-22eaa41bcdc9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.122222 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 195/390] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:10.122840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27139097-f1b5-4372-86a5-22eaa41bcdc9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.130316 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 196/391] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Oct 24 08:35:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:10.131456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2995e-b19a-4049-ad6b-568101e0f338 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.230236 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2995e-b19a-4049-ad6b-568101e0f338 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.233568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27139097-f1b5-4372-86a5-22eaa41bcdc9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.262800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dd4df-6926-44aa-9924-6e029313ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.263414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dd4df-6926-44aa-9924-6e029313ffe4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.263694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dd4df-6926-44aa-9924-6e029313ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.263966 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dd4df-6926-44aa-9924-6e029313ffe4 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.264612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dd4df-6926-44aa-9924-6e029313ffe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:10.264918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dd4df-6926-44aa-9924-6e029313ffe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.266306 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028dd4df-6926-44aa-9924-6e029313ffe4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.270042 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bddae7-d6a7-4584-becc-1fee8a960b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.270670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bddae7-d6a7-4584-becc-1fee8a960b64 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.270966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bddae7-d6a7-4584-becc-1fee8a960b64 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.271244 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bddae7-d6a7-4584-becc-1fee8a960b64 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.271967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bddae7-d6a7-4584-becc-1fee8a960b64 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:10.272281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bddae7-d6a7-4584-becc-1fee8a960b64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.273701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bddae7-d6a7-4584-becc-1fee8a960b64 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.277749 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 197/392] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:10.278569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dd4df-6926-44aa-9924-6e029313ffe4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.311340 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 196/393] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:10.312193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bddae7-d6a7-4584-becc-1fee8a960b64 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.326549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:10.339105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:10.377982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dd4df-6926-44aa-9924-6e029313ffe4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.409494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bddae7-d6a7-4584-becc-1fee8a960b64 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.412011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ad547-176b-40fd-9159-ae67ecfc2ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.412647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ad547-176b-40fd-9159-ae67ecfc2ce9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.412926 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ad547-176b-40fd-9159-ae67ecfc2ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.413197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ad547-176b-40fd-9159-ae67ecfc2ce9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.423706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ad547-176b-40fd-9159-ae67ecfc2ce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.425223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ad547-176b-40fd-9159-ae67ecfc2ce9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a39ad547-176b-40fd-9159-ae67ecfc2ce9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:10.439389 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:10.440304 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:10.441297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:10.441297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:10.441297 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:10.441297 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 198/394] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:10.441733 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ad547-176b-40fd-9159-ae67ecfc2ce9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.446028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6f963-8b3a-4e42-874d-38fbf232c46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.446658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6f963-8b3a-4e42-874d-38fbf232c46f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.446967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6f963-8b3a-4e42-874d-38fbf232c46f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.447239 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6f963-8b3a-4e42-874d-38fbf232c46f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.457387 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6f963-8b3a-4e42-874d-38fbf232c46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.458871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6f963-8b3a-4e42-874d-38fbf232c46f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3ff6f963-8b3a-4e42-874d-38fbf232c46f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:10.473081 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:10.474026 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:10.475303 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:10.475303 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:10.475303 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:10.475303 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 197/395] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:10.475303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6f963-8b3a-4e42-874d-38fbf232c46f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.542534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:10.545639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ad547-176b-40fd-9159-ae67ecfc2ce9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.555286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:10.578671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6f963-8b3a-4e42-874d-38fbf232c46f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.580548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ae735-fe64-40c6-83e3-59400bf1c536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.581261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ae735-fe64-40c6-83e3-59400bf1c536 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.581607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ae735-fe64-40c6-83e3-59400bf1c536 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.581948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ae735-fe64-40c6-83e3-59400bf1c536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.592343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ae735-fe64-40c6-83e3-59400bf1c536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.593796 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ae735-fe64-40c6-83e3-59400bf1c536 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ea7ae735-fe64-40c6-83e3-59400bf1c536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:10.608063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:10.609003 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:10.609853 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:10.609853 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:10.609853 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:10.610992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763d6f8-e2e7-4c93-b896-56225c5f0687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.611609 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763d6f8-e2e7-4c93-b896-56225c5f0687 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.611880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763d6f8-e2e7-4c93-b896-56225c5f0687 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.612151 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763d6f8-e2e7-4c93-b896-56225c5f0687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.614288 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 199/396] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:10.615027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ae735-fe64-40c6-83e3-59400bf1c536 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.618751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763d6f8-e2e7-4c93-b896-56225c5f0687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.620194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763d6f8-e2e7-4c93-b896-56225c5f0687 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-a763d6f8-e2e7-4c93-b896-56225c5f0687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:10.631521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:10.632365 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 198/397] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:10.633241 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763d6f8-e2e7-4c93-b896-56225c5f0687 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.714113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ae735-fe64-40c6-83e3-59400bf1c536 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.730894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763d6f8-e2e7-4c93-b896-56225c5f0687 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.747385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c3836-b3c5-4bbb-8035-fc02fb7a112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.748021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c3836-b3c5-4bbb-8035-fc02fb7a112f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.748310 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c3836-b3c5-4bbb-8035-fc02fb7a112f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.748562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c3836-b3c5-4bbb-8035-fc02fb7a112f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.758950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c3836-b3c5-4bbb-8035-fc02fb7a112f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.760406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c3836-b3c5-4bbb-8035-fc02fb7a112f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.767453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdcb89-a23a-4c2f-a8ee-087004877c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.768051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdcb89-a23a-4c2f-a8ee-087004877c5a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.768329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdcb89-a23a-4c2f-a8ee-087004877c5a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.768596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdcb89-a23a-4c2f-a8ee-087004877c5a None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.769232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdcb89-a23a-4c2f-a8ee-087004877c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:10.769529 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdcb89-a23a-4c2f-a8ee-087004877c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.770941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdcb89-a23a-4c2f-a8ee-087004877c5a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7c1c3836-b3c5-4bbb-8035-fc02fb7a112f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:10.774001 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:10.775052 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:10.775994 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:10.775994 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:10.775994 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:10.775994 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 200/398] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:10.775994 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c3836-b3c5-4bbb-8035-fc02fb7a112f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.778031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:10.783070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:10.788380 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 199/399] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:10.789240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdcb89-a23a-4c2f-a8ee-087004877c5a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.877612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c3836-b3c5-4bbb-8035-fc02fb7a112f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.886329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdcb89-a23a-4c2f-a8ee-087004877c5a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:10.910867 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54b7c4-ac8b-4d84-bb17-81b0d422eee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.911831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54b7c4-ac8b-4d84-bb17-81b0d422eee8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.912111 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54b7c4-ac8b-4d84-bb17-81b0d422eee8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.912391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54b7c4-ac8b-4d84-bb17-81b0d422eee8 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.913053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54b7c4-ac8b-4d84-bb17-81b0d422eee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:10.913358 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54b7c4-ac8b-4d84-bb17-81b0d422eee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.914772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54b7c4-ac8b-4d84-bb17-81b0d422eee8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.920581 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a42cad-8c63-42c4-a041-881096579764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:10.921174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a42cad-8c63-42c4-a041-881096579764 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:10.921454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a42cad-8c63-42c4-a041-881096579764 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:10.921722 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a42cad-8c63-42c4-a041-881096579764 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:10.922351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a42cad-8c63-42c4-a041-881096579764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:10.922650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a42cad-8c63-42c4-a041-881096579764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:10.924052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a42cad-8c63-42c4-a041-881096579764 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:10.931042 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 201/400] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:10.931042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54b7c4-ac8b-4d84-bb17-81b0d422eee8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.940309 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 200/401] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:10.940309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a42cad-8c63-42c4-a041-881096579764 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:10.992571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:10.996221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:11.030014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54b7c4-ac8b-4d84-bb17-81b0d422eee8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.038522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a42cad-8c63-42c4-a041-881096579764 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.063138 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056d6a-9f58-42c1-8d52-a037b2bfe79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.063752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056d6a-9f58-42c1-8d52-a037b2bfe79b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.064037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056d6a-9f58-42c1-8d52-a037b2bfe79b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.064333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056d6a-9f58-42c1-8d52-a037b2bfe79b None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.065022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7056d6a-9f58-42c1-8d52-a037b2bfe79b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:11.065328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7056d6a-9f58-42c1-8d52-a037b2bfe79b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.066787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7056d6a-9f58-42c1-8d52-a037b2bfe79b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.071725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd2159-9b57-4db6-8827-24d8b49c5745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.072328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd2159-9b57-4db6-8827-24d8b49c5745 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.072599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd2159-9b57-4db6-8827-24d8b49c5745 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.072863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd2159-9b57-4db6-8827-24d8b49c5745 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.073491 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd2159-9b57-4db6-8827-24d8b49c5745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:11.073786 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd2159-9b57-4db6-8827-24d8b49c5745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.075173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd2159-9b57-4db6-8827-24d8b49c5745 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.083794 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 202/402] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:11.084562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056d6a-9f58-42c1-8d52-a037b2bfe79b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.092295 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 201/403] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:11.096710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd2159-9b57-4db6-8827-24d8b49c5745 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.181778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056d6a-9f58-42c1-8d52-a037b2bfe79b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.191592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd2159-9b57-4db6-8827-24d8b49c5745 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.204312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:11.212301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:11.216610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2503eb-6ae1-4d0b-80c2-71a2483dc58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.217250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2503eb-6ae1-4d0b-80c2-71a2483dc58b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.217538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2503eb-6ae1-4d0b-80c2-71a2483dc58b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.217803 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2503eb-6ae1-4d0b-80c2-71a2483dc58b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.224797 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2503eb-6ae1-4d0b-80c2-71a2483dc58b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.226252 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2503eb-6ae1-4d0b-80c2-71a2483dc58b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.229147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62999338-de90-4b53-a0ee-e98066e67bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.229772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62999338-de90-4b53-a0ee-e98066e67bb1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.230046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62999338-de90-4b53-a0ee-e98066e67bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.230376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62999338-de90-4b53-a0ee-e98066e67bb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.236925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62999338-de90-4b53-a0ee-e98066e67bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.238360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62999338-de90-4b53-a0ee-e98066e67bb1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-8e2503eb-6ae1-4d0b-80c2-71a2483dc58b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:11.239026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:11.239991 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 203/404] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:11.240897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2503eb-6ae1-4d0b-80c2-71a2483dc58b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-62999338-de90-4b53-a0ee-e98066e67bb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:11.253883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:11.254951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 202/405] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:11.255820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62999338-de90-4b53-a0ee-e98066e67bb1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.344725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2503eb-6ae1-4d0b-80c2-71a2483dc58b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.354661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62999338-de90-4b53-a0ee-e98066e67bb1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.378824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c331e-0dd5-48df-bbe7-c256cd62689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.379442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c331e-0dd5-48df-bbe7-c256cd62689f None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.379719 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c331e-0dd5-48df-bbe7-c256cd62689f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.379975 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c331e-0dd5-48df-bbe7-c256cd62689f None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.380641 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c331e-0dd5-48df-bbe7-c256cd62689f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.382097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c331e-0dd5-48df-bbe7-c256cd62689f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.387703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97b4af-1757-4b1b-98bf-16dd7e12906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.388352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97b4af-1757-4b1b-98bf-16dd7e12906a None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.388628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97b4af-1757-4b1b-98bf-16dd7e12906a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.388899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97b4af-1757-4b1b-98bf-16dd7e12906a None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/f257d9b665714bff8b4ceb4fc5dc8387/roles/cb7d27022ded4ea2afd3d5ebaefbf429` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.402940 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 204/406] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:35:11 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:11.403780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c331e-0dd5-48df-bbe7-c256cd62689f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.404537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97b4af-1757-4b1b-98bf-16dd7e12906a None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=f257d9b665714bff8b4ceb4fc5dc8387, role_id=cb7d27022ded4ea2afd3d5ebaefbf429)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.405998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97b4af-1757-4b1b-98bf-16dd7e12906a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.425910 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 203/407] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:35:11 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/f257d9b665714bff8b4ceb4fc5dc8387/roles/cb7d27022ded4ea2afd3d5ebaefbf429 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:35:11.431770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97b4af-1757-4b1b-98bf-16dd7e12906a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.470174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:11.492942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:11.581665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97b4af-1757-4b1b-98bf-16dd7e12906a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.595977 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c331e-0dd5-48df-bbe7-c256cd62689f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.635390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4164288-c195-4bbc-81c0-aa73df5adf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.636015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4164288-c195-4bbc-81c0-aa73df5adf58 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.636287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4164288-c195-4bbc-81c0-aa73df5adf58 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.636596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4164288-c195-4bbc-81c0-aa73df5adf58 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.637330 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4164288-c195-4bbc-81c0-aa73df5adf58 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:11.637639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4164288-c195-4bbc-81c0-aa73df5adf58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.639095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4164288-c195-4bbc-81c0-aa73df5adf58 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.647668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283202e1-2fb9-4330-be98-43a5754bb5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.648427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283202e1-2fb9-4330-be98-43a5754bb5d2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.648712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283202e1-2fb9-4330-be98-43a5754bb5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.648987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283202e1-2fb9-4330-be98-43a5754bb5d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.656662 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283202e1-2fb9-4330-be98-43a5754bb5d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.658139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283202e1-2fb9-4330-be98-43a5754bb5d2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.662845 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 204/408] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Oct 24 08:35:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:11.662845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4164288-c195-4bbc-81c0-aa73df5adf58 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-283202e1-2fb9-4330-be98-43a5754bb5d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:11.670997 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:11.671964 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 205/409] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:11.673758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283202e1-2fb9-4330-be98-43a5754bb5d2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.752067 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:11.769116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4164288-c195-4bbc-81c0-aa73df5adf58 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.799719 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:11.805557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283202e1-2fb9-4330-be98-43a5754bb5d2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.814470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4da749-49e2-4b16-916f-dd20a2bff8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.815260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4da749-49e2-4b16-916f-dd20a2bff8ea None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.815637 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4da749-49e2-4b16-916f-dd20a2bff8ea None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.816005 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4da749-49e2-4b16-916f-dd20a2bff8ea None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.816813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4da749-49e2-4b16-916f-dd20a2bff8ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:11.817239 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4da749-49e2-4b16-916f-dd20a2bff8ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.818825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4da749-49e2-4b16-916f-dd20a2bff8ea None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.839151 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 205/410] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:11.845146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4da749-49e2-4b16-916f-dd20a2bff8ea None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.846558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197af47b-8b1f-4a03-bc2c-b88a1d3f9777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.847302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197af47b-8b1f-4a03-bc2c-b88a1d3f9777 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.847603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197af47b-8b1f-4a03-bc2c-b88a1d3f9777 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.847867 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197af47b-8b1f-4a03-bc2c-b88a1d3f9777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.863131 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197af47b-8b1f-4a03-bc2c-b88a1d3f9777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.864870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197af47b-8b1f-4a03-bc2c-b88a1d3f9777 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-197af47b-8b1f-4a03-bc2c-b88a1d3f9777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:11.884227 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:11.885257 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:11.886294 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:11.886294 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:11.886294 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:11.886294 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 206/411] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:11.887672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197af47b-8b1f-4a03-bc2c-b88a1d3f9777 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:11.951661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4da749-49e2-4b16-916f-dd20a2bff8ea None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:11.995004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d73d8-d348-4c05-91ae-18fff7c88af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:11.995820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d73d8-d348-4c05-91ae-18fff7c88af3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:11.996217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d73d8-d348-4c05-91ae-18fff7c88af3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:11.996580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d73d8-d348-4c05-91ae-18fff7c88af3 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:11.997381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d73d8-d348-4c05-91ae-18fff7c88af3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:11.997780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d73d8-d348-4c05-91ae-18fff7c88af3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:11.999996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d73d8-d348-4c05-91ae-18fff7c88af3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:12.014189 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 206/412] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:12.015349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d73d8-d348-4c05-91ae-18fff7c88af3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:12.018342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197af47b-8b1f-4a03-bc2c-b88a1d3f9777 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:12.025081 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:12.054034 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc3f07-180a-46ff-9068-3eee93838249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:12.055510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc3f07-180a-46ff-9068-3eee93838249 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:12.055510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc3f07-180a-46ff-9068-3eee93838249 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:12.055510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc3f07-180a-46ff-9068-3eee93838249 None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/6046d6440af84aa28aab47a018638e18/roles/cb7d27022ded4ea2afd3d5ebaefbf429` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:12.070678 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc3f07-180a-46ff-9068-3eee93838249 None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=6046d6440af84aa28aab47a018638e18, role_id=cb7d27022ded4ea2afd3d5ebaefbf429)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:12.072244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc3f07-180a-46ff-9068-3eee93838249 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:12.094019 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 207/413] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:35:11 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/6046d6440af84aa28aab47a018638e18/roles/cb7d27022ded4ea2afd3d5ebaefbf429 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:35:12.094019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc3f07-180a-46ff-9068-3eee93838249 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:12.101276 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:12.167508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d73d8-d348-4c05-91ae-18fff7c88af3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:12.222965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc3f07-180a-46ff-9068-3eee93838249 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:12.261844 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64430ae6-b560-44db-9d75-94b3fd235222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:12.262603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64430ae6-b560-44db-9d75-94b3fd235222 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:12.262913 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64430ae6-b560-44db-9d75-94b3fd235222 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:12.263189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64430ae6-b560-44db-9d75-94b3fd235222 None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/3b1c2ea68e4741728469ab7cc542b6d5/roles/cb7d27022ded4ea2afd3d5ebaefbf429` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:12.281769 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64430ae6-b560-44db-9d75-94b3fd235222 None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=3b1c2ea68e4741728469ab7cc542b6d5, role_id=cb7d27022ded4ea2afd3d5ebaefbf429)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:12.283197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64430ae6-b560-44db-9d75-94b3fd235222 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:12.301224 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 208/414] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:35:12 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/3b1c2ea68e4741728469ab7cc542b6d5/roles/cb7d27022ded4ea2afd3d5ebaefbf429 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:35:12.306279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cfd45-23b8-4a13-9a1b-67795de0203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:12.306974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cfd45-23b8-4a13-9a1b-67795de0203a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:12.307255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cfd45-23b8-4a13-9a1b-67795de0203a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:12.307526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cfd45-23b8-4a13-9a1b-67795de0203a None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:12.308283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cfd45-23b8-4a13-9a1b-67795de0203a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:12.308594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cfd45-23b8-4a13-9a1b-67795de0203a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:12.310032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cfd45-23b8-4a13-9a1b-67795de0203a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:12.344865 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 207/415] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:12.350979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64430ae6-b560-44db-9d75-94b3fd235222 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:12.431186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:12.488110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64430ae6-b560-44db-9d75-94b3fd235222 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:12.538095 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64106802-1310-4cb4-85b3-3df8e51b15fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:12.539591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64106802-1310-4cb4-85b3-3df8e51b15fd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:12.540126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64106802-1310-4cb4-85b3-3df8e51b15fd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:12.540593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64106802-1310-4cb4-85b3-3df8e51b15fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:12.549883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64106802-1310-4cb4-85b3-3df8e51b15fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:12.551662 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64106802-1310-4cb4-85b3-3df8e51b15fd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-64106802-1310-4cb4-85b3-3df8e51b15fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:12.568206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:12.569084 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:12.569984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:12.569984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:12.569984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:12.569984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:12.569984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:12.569984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:12.569984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:12.569984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:12.572344 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 209/416] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:12.581408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cfd45-23b8-4a13-9a1b-67795de0203a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:12.651139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:12.689917 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cfd45-23b8-4a13-9a1b-67795de0203a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:12.730309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3622cf-6c86-4bff-a94b-3241b71b4865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:12.731319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3622cf-6c86-4bff-a94b-3241b71b4865 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:12.731812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3622cf-6c86-4bff-a94b-3241b71b4865 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:12.732277 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3622cf-6c86-4bff-a94b-3241b71b4865 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:12.733164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3622cf-6c86-4bff-a94b-3241b71b4865 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:12.733753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3622cf-6c86-4bff-a94b-3241b71b4865 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:12.735504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3622cf-6c86-4bff-a94b-3241b71b4865 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:12.750662 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 208/417] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:12.756532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64106802-1310-4cb4-85b3-3df8e51b15fd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:12.874007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64106802-1310-4cb4-85b3-3df8e51b15fd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:12.888624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:12.910572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f000e-0c2f-4af2-9e82-013a7d2b4596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:12.911676 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f000e-0c2f-4af2-9e82-013a7d2b4596 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:12.911951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f000e-0c2f-4af2-9e82-013a7d2b4596 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:12.912219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f000e-0c2f-4af2-9e82-013a7d2b4596 None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/2254623f523c48f0aa454b85be41df95/roles/cb7d27022ded4ea2afd3d5ebaefbf429` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:12.926811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f000e-0c2f-4af2-9e82-013a7d2b4596 None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=2254623f523c48f0aa454b85be41df95, role_id=cb7d27022ded4ea2afd3d5ebaefbf429)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:12.928270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f000e-0c2f-4af2-9e82-013a7d2b4596 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:12.944157 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 210/418] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:35:12 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/2254623f523c48f0aa454b85be41df95/roles/cb7d27022ded4ea2afd3d5ebaefbf429 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:35:13.858436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdae319-047b-4cc2-abd7-5757994982f3 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:13.859416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdae319-047b-4cc2-abd7-5757994982f3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:13.859854 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdae319-047b-4cc2-abd7-5757994982f3 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:13.861458 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 209/419] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:13.867706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8eccc-615a-4ea3-8d83-5e88541ba5e2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:13.868247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8eccc-615a-4ea3-8d83-5e88541ba5e2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:13.868247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8eccc-615a-4ea3-8d83-5e88541ba5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:13.894907 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-30b8eccc-615a-4ea3-8d83-5e88541ba5e2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:13.896128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-30b8eccc-615a-4ea3-8d83-5e88541ba5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:13.968609 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816351c-4013-4e6b-a496-454f8475daf2 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:13.968986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816351c-4013-4e6b-a496-454f8475daf2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:13.969278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816351c-4013-4e6b-a496-454f8475daf2 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:13.971593 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 210/420] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:35:13.977876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361dbdd-fd0f-4c5d-af40-d9428bcbe43d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:13.978166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361dbdd-fd0f-4c5d-af40-d9428bcbe43d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:13.978447 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361dbdd-fd0f-4c5d-af40-d9428bcbe43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:13.997116 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 211/421] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:14.027842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1361dbdd-fd0f-4c5d-af40-d9428bcbe43d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:14.029373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1361dbdd-fd0f-4c5d-af40-d9428bcbe43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:14.131760 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 211/422] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:14.300867 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8eccc-615a-4ea3-8d83-5e88541ba5e2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:14.418265 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:14.426886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8eccc-615a-4ea3-8d83-5e88541ba5e2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:14.470487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814c68c-9899-4ae8-bcbd-c7ff10317e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:14.471155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814c68c-9899-4ae8-bcbd-c7ff10317e9f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:14.471430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814c68c-9899-4ae8-bcbd-c7ff10317e9f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:14.471698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814c68c-9899-4ae8-bcbd-c7ff10317e9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:14.479377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361dbdd-fd0f-4c5d-af40-d9428bcbe43d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:14.485481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814c68c-9899-4ae8-bcbd-c7ff10317e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7814c68c-9899-4ae8-bcbd-c7ff10317e9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:14.490315 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:14.491244 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:14.492042 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 212/423] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:14.497731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814c68c-9899-4ae8-bcbd-c7ff10317e9f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:14.594710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:14.678813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361dbdd-fd0f-4c5d-af40-d9428bcbe43d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:14.693953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814c68c-9899-4ae8-bcbd-c7ff10317e9f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:14.749683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b58152-c590-4215-838b-8e5e611d63a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:14.750312 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b58152-c590-4215-838b-8e5e611d63a4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:14.750584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b58152-c590-4215-838b-8e5e611d63a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:14.750879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b58152-c590-4215-838b-8e5e611d63a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:14.762346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccceeb9-1d86-4c09-b4d4-8bad0d31ae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:14.762989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccceeb9-1d86-4c09-b4d4-8bad0d31ae27 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:14.763267 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccceeb9-1d86-4c09-b4d4-8bad0d31ae27 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:14.763540 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccceeb9-1d86-4c09-b4d4-8bad0d31ae27 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:14.764203 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccceeb9-1d86-4c09-b4d4-8bad0d31ae27 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:14.764501 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccceeb9-1d86-4c09-b4d4-8bad0d31ae27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:14.765891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccceeb9-1d86-4c09-b4d4-8bad0d31ae27 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:14.775211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b58152-c590-4215-838b-8e5e611d63a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:14.783520 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 213/424] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-07b58152-c590-4215-838b-8e5e611d63a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:14.783768 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:14.784653 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:14.785440 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 212/425] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:14.791738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccceeb9-1d86-4c09-b4d4-8bad0d31ae27 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:14.796517 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b58152-c590-4215-838b-8e5e611d63a4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:14.829284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:14.938743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b58152-c590-4215-838b-8e5e611d63a4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:14.956339 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:14.978312 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58287dcf-a912-4a63-b3cb-bad9b5add409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:14.978963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58287dcf-a912-4a63-b3cb-bad9b5add409 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:14.979243 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58287dcf-a912-4a63-b3cb-bad9b5add409 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:14.979513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58287dcf-a912-4a63-b3cb-bad9b5add409 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:14.980314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58287dcf-a912-4a63-b3cb-bad9b5add409 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:14.980619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58287dcf-a912-4a63-b3cb-bad9b5add409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:14.982047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58287dcf-a912-4a63-b3cb-bad9b5add409 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:14.991541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccceeb9-1d86-4c09-b4d4-8bad0d31ae27 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:14.992071 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 213/426] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:14.993367 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628ca73-6f66-4207-bff8-5d41e93f9148 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:14.993651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628ca73-6f66-4207-bff8-5d41e93f9148 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:14.994000 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628ca73-6f66-4207-bff8-5d41e93f9148 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:14.999889 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 214/427] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:15.000904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628ca73-6f66-4207-bff8-5d41e93f9148 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.033725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cc6ca-71ac-4ca3-a17d-42f30114dcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.034555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cc6ca-71ac-4ca3-a17d-42f30114dcf8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.034944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cc6ca-71ac-4ca3-a17d-42f30114dcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.035364 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cc6ca-71ac-4ca3-a17d-42f30114dcf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.047046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cc6ca-71ac-4ca3-a17d-42f30114dcf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:15.048769 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cc6ca-71ac-4ca3-a17d-42f30114dcf8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ed7cc6ca-71ac-4ca3-a17d-42f30114dcf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:15.063984 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:15.064918 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:15.065779 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:15.065779 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:15.065779 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:15.065779 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 214/428] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:15.067283 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c46bf-7437-421d-876f-7e3cbb428814 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.067709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c46bf-7437-421d-876f-7e3cbb428814 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.068069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c46bf-7437-421d-876f-7e3cbb428814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.089115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-873c46bf-7437-421d-876f-7e3cbb428814 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:15.090343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-873c46bf-7437-421d-876f-7e3cbb428814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:15.111960 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628ca73-6f66-4207-bff8-5d41e93f9148 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:15.150549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:15.153370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88cb5e-9c03-4020-8edc-41dd5f626abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.154001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88cb5e-9c03-4020-8edc-41dd5f626abc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.154278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88cb5e-9c03-4020-8edc-41dd5f626abc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.154568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88cb5e-9c03-4020-8edc-41dd5f626abc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.159107 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 215/429] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:15.163509 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c46bf-7437-421d-876f-7e3cbb428814 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.166336 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88cb5e-9c03-4020-8edc-41dd5f626abc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:15.167902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88cb5e-9c03-4020-8edc-41dd5f626abc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-2f88cb5e-9c03-4020-8edc-41dd5f626abc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:15.183412 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:15.184391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:15.185238 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:15.185238 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:15.185238 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:15.185238 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 215/430] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:15.185877 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb7d34-6547-48a6-9620-594321d27dce None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.185969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb7d34-6547-48a6-9620-594321d27dce None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.186213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb7d34-6547-48a6-9620-594321d27dce None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.191857 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 216/431] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:15.192907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804c969-413a-407f-b8c7-cb44709c9505 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.193188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804c969-413a-407f-b8c7-cb44709c9505 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.193455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804c969-413a-407f-b8c7-cb44709c9505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.211789 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7804c969-413a-407f-b8c7-cb44709c9505 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:15.213015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7804c969-413a-407f-b8c7-cb44709c9505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:15.242287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:15.295594 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 217/432] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:15.297482 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804c969-413a-407f-b8c7-cb44709c9505 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.305598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c46bf-7437-421d-876f-7e3cbb428814 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:15.395728 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed44ff7-46e6-4f9c-b232-37d79be76dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.396321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed44ff7-46e6-4f9c-b232-37d79be76dcf None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.396590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed44ff7-46e6-4f9c-b232-37d79be76dcf None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.396861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed44ff7-46e6-4f9c-b232-37d79be76dcf None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.397504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed44ff7-46e6-4f9c-b232-37d79be76dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:15.397804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed44ff7-46e6-4f9c-b232-37d79be76dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:15.399467 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed44ff7-46e6-4f9c-b232-37d79be76dcf None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:15.409006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804c969-413a-407f-b8c7-cb44709c9505 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:15.424365 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 216/433] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:15.425359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed44ff7-46e6-4f9c-b232-37d79be76dcf None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.457929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c44dac-90ad-4bc5-a85d-b426271e0c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.458516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c44dac-90ad-4bc5-a85d-b426271e0c69 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.458831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c44dac-90ad-4bc5-a85d-b426271e0c69 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.459103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c44dac-90ad-4bc5-a85d-b426271e0c69 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.459781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c44dac-90ad-4bc5-a85d-b426271e0c69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:15.460075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c44dac-90ad-4bc5-a85d-b426271e0c69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:15.461531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c44dac-90ad-4bc5-a85d-b426271e0c69 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:15.474595 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 218/434] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:15.475554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c44dac-90ad-4bc5-a85d-b426271e0c69 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.498070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:15.501560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:15.559786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed44ff7-46e6-4f9c-b232-37d79be76dcf None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:15.586015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c44dac-90ad-4bc5-a85d-b426271e0c69 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:15.593933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f1a91-b79a-439b-b3d2-7b186e454227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.594569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f1a91-b79a-439b-b3d2-7b186e454227 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.594881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f1a91-b79a-439b-b3d2-7b186e454227 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.595153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f1a91-b79a-439b-b3d2-7b186e454227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.605250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f1a91-b79a-439b-b3d2-7b186e454227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-b87f1a91-b79a-439b-b3d2-7b186e454227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:15.610514 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:15.611494 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:15.612303 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 217/435] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:15.613126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f1a91-b79a-439b-b3d2-7b186e454227 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.623049 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d693af0-7145-498e-b8dc-7ba89418ba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.623691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d693af0-7145-498e-b8dc-7ba89418ba3d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.623964 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d693af0-7145-498e-b8dc-7ba89418ba3d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.624255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d693af0-7145-498e-b8dc-7ba89418ba3d None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.631697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d693af0-7145-498e-b8dc-7ba89418ba3d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:15.633160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d693af0-7145-498e-b8dc-7ba89418ba3d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-2d693af0-7145-498e-b8dc-7ba89418ba3d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:15.644474 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:15.645373 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 219/436] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Oct 24 08:35:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49547496-2aaa-4d7d-94c9-0c3c6fa1049a None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.651062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49547496-2aaa-4d7d-94c9-0c3c6fa1049a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.651790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49547496-2aaa-4d7d-94c9-0c3c6fa1049a None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.651790 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 220/437] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:15.651790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49547496-2aaa-4d7d-94c9-0c3c6fa1049a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.716141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f1a91-b79a-439b-b3d2-7b186e454227 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:15.728690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:15.732297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:15.749236 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49547496-2aaa-4d7d-94c9-0c3c6fa1049a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:15.750019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc19cb-c4a5-4010-ac63-d1227248df9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.750650 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc19cb-c4a5-4010-ac63-d1227248df9f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.750962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc19cb-c4a5-4010-ac63-d1227248df9f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.751225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc19cb-c4a5-4010-ac63-d1227248df9f None admin] PATH_INFO: `/v3/users/swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.758563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc19cb-c4a5-4010-ac63-d1227248df9f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:15.760046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc19cb-c4a5-4010-ac63-d1227248df9f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ebbc19cb-c4a5-4010-ac63-d1227248df9f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:15.771100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:15.772014 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 218/438] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:35:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:15.772905 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc19cb-c4a5-4010-ac63-d1227248df9f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.784779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677bac-356d-494d-b104-c17f97c8a4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.785479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677bac-356d-494d-b104-c17f97c8a4d1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.785761 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677bac-356d-494d-b104-c17f97c8a4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.786030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677bac-356d-494d-b104-c17f97c8a4d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.797511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4677bac-356d-494d-b104-c17f97c8a4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c4677bac-356d-494d-b104-c17f97c8a4d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:15.802268 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:15.803304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:15.804121 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 221/439] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:15.804775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677bac-356d-494d-b104-c17f97c8a4d1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.874472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc19cb-c4a5-4010-ac63-d1227248df9f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:15.901162 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677bac-356d-494d-b104-c17f97c8a4d1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:15.906288 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21669b0-0a07-4e8e-a26a-0356647406a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.906927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21669b0-0a07-4e8e-a26a-0356647406a8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.907205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21669b0-0a07-4e8e-a26a-0356647406a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.907474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21669b0-0a07-4e8e-a26a-0356647406a8 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.908131 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21669b0-0a07-4e8e-a26a-0356647406a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:15.908427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21669b0-0a07-4e8e-a26a-0356647406a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:15.909799 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21669b0-0a07-4e8e-a26a-0356647406a8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:15.917976 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 219/440] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:15.919223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879bd94-1229-40f6-aad0-e4d4bfc4a461 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.919512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879bd94-1229-40f6-aad0-e4d4bfc4a461 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.919786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879bd94-1229-40f6-aad0-e4d4bfc4a461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.937533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fc9b3-52a2-4f39-988d-af773f90e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:15.938088 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0879bd94-1229-40f6-aad0-e4d4bfc4a461 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:15.938196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fc9b3-52a2-4f39-988d-af773f90e796 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:15.938401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fc9b3-52a2-4f39-988d-af773f90e796 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:15.938668 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fc9b3-52a2-4f39-988d-af773f90e796 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:15.939303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0879bd94-1229-40f6-aad0-e4d4bfc4a461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:15.939402 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fc9b3-52a2-4f39-988d-af773f90e796 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:15.939843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fc9b3-52a2-4f39-988d-af773f90e796 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:15.941101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fc9b3-52a2-4f39-988d-af773f90e796 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:15.951397 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:15.959958 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 222/441] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Oct 24 08:35:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:15.960609 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fc9b3-52a2-4f39-988d-af773f90e796 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:15.974037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:16.012216 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 220/442] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:16.013073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879bd94-1229-40f6-aad0-e4d4bfc4a461 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.064488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fc9b3-52a2-4f39-988d-af773f90e796 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.101809 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21196dd0-e335-46d4-ab20-76bfcc91acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.102732 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21196dd0-e335-46d4-ab20-76bfcc91acb0 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.103140 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21196dd0-e335-46d4-ab20-76bfcc91acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.103546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21196dd0-e335-46d4-ab20-76bfcc91acb0 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.104903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21196dd0-e335-46d4-ab20-76bfcc91acb0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:16.105588 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21196dd0-e335-46d4-ab20-76bfcc91acb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.107830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21196dd0-e335-46d4-ab20-76bfcc91acb0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.115738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879bd94-1229-40f6-aad0-e4d4bfc4a461 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.129885 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 223/443] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Oct 24 08:35:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:16.131161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21196dd0-e335-46d4-ab20-76bfcc91acb0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.152640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a591d7-56c6-462d-9aff-02b3d7079f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.153289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a591d7-56c6-462d-9aff-02b3d7079f66 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.153671 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a591d7-56c6-462d-9aff-02b3d7079f66 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.153956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a591d7-56c6-462d-9aff-02b3d7079f66 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.154642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a591d7-56c6-462d-9aff-02b3d7079f66 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:16.154981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a591d7-56c6-462d-9aff-02b3d7079f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.156357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a591d7-56c6-462d-9aff-02b3d7079f66 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.165062 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 221/444] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:16.165829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a591d7-56c6-462d-9aff-02b3d7079f66 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.195470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:16.198358 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:16.245173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21196dd0-e335-46d4-ab20-76bfcc91acb0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.263608 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a591d7-56c6-462d-9aff-02b3d7079f66 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.282227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be45b1-4d87-4a3e-bc13-df970867937f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.282892 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be45b1-4d87-4a3e-bc13-df970867937f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.283179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be45b1-4d87-4a3e-bc13-df970867937f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.283442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be45b1-4d87-4a3e-bc13-df970867937f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.293762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be45b1-4d87-4a3e-bc13-df970867937f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.295293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be45b1-4d87-4a3e-bc13-df970867937f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.300060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a84c77-425a-44b3-8197-c7194e0909ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.300668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a84c77-425a-44b3-8197-c7194e0909ba None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.300943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a84c77-425a-44b3-8197-c7194e0909ba None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.301212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a84c77-425a-44b3-8197-c7194e0909ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-81be45b1-4d87-4a3e-bc13-df970867937f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:16.309567 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:16.310584 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:16.312276 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:16.312276 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:16.312276 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:16.312276 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 224/445] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:16.312276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be45b1-4d87-4a3e-bc13-df970867937f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.366935 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a84c77-425a-44b3-8197-c7194e0909ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.368391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a84c77-425a-44b3-8197-c7194e0909ba None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.395657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:16.407264 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be45b1-4d87-4a3e-bc13-df970867937f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.416194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:16.443107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cc32a-5718-4226-af8e-21607f3dd8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.443720 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cc32a-5718-4226-af8e-21607f3dd8cb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.443993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cc32a-5718-4226-af8e-21607f3dd8cb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.444266 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cc32a-5718-4226-af8e-21607f3dd8cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.454390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cc32a-5718-4226-af8e-21607f3dd8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.455943 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cc32a-5718-4226-af8e-21607f3dd8cb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.463461 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 222/446] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:35:16.464285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a84c77-425a-44b3-8197-c7194e0909ba None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-559cc32a-5718-4226-af8e-21607f3dd8cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:16.469462 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:16.470392 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:16.471263 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:16.471263 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:16.471263 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:16.474944 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 225/447] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:16.475738 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cc32a-5718-4226-af8e-21607f3dd8cb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.562099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a84c77-425a-44b3-8197-c7194e0909ba None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.572638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cc32a-5718-4226-af8e-21607f3dd8cb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.590087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:16.596239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307fe61-db57-427d-9917-a4c5ab300925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.596871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307fe61-db57-427d-9917-a4c5ab300925 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.597141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307fe61-db57-427d-9917-a4c5ab300925 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.597410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307fe61-db57-427d-9917-a4c5ab300925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.605727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ba33e-43e8-45bc-a2ba-75d352144bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.606353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ba33e-43e8-45bc-a2ba-75d352144bf3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.606631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ba33e-43e8-45bc-a2ba-75d352144bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.606928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ba33e-43e8-45bc-a2ba-75d352144bf3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.607293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307fe61-db57-427d-9917-a4c5ab300925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.608741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307fe61-db57-427d-9917-a4c5ab300925 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.616662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ba33e-43e8-45bc-a2ba-75d352144bf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d11ba33e-43e8-45bc-a2ba-75d352144bf3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:16.621762 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:16.622643 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:16.623478 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-b307fe61-db57-427d-9917-a4c5ab300925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:16.624076 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:16.624871 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:16.625752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:16.625752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:16.625752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:16.625994 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 226/448] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:16.625994 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ba33e-43e8-45bc-a2ba-75d352144bf3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.626111 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 223/449] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:16.626111 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307fe61-db57-427d-9917-a4c5ab300925 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.649560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:16.722019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307fe61-db57-427d-9917-a4c5ab300925 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.724683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ba33e-43e8-45bc-a2ba-75d352144bf3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.754113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e85ee-7c94-4665-9663-ef60af596098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.754763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e85ee-7c94-4665-9663-ef60af596098 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.755042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e85ee-7c94-4665-9663-ef60af596098 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.755311 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e85ee-7c94-4665-9663-ef60af596098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.756054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aaef9-c7d9-4d3d-b9c1-95f94911d5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.756646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aaef9-c7d9-4d3d-b9c1-95f94911d5fc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.756945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aaef9-c7d9-4d3d-b9c1-95f94911d5fc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.757208 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aaef9-c7d9-4d3d-b9c1-95f94911d5fc None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.757844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aaef9-c7d9-4d3d-b9c1-95f94911d5fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:16.758144 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aaef9-c7d9-4d3d-b9c1-95f94911d5fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.759551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aaef9-c7d9-4d3d-b9c1-95f94911d5fc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.765573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e85ee-7c94-4665-9663-ef60af596098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.767033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e85ee-7c94-4665-9663-ef60af596098 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.770873 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 227/450] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:16.771647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aaef9-c7d9-4d3d-b9c1-95f94911d5fc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-012e85ee-7c94-4665-9663-ef60af596098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:16.781734 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:16.782652 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:16.783527 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:16.783527 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:16.783527 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:16.783527 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 224/451] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:16.783752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e85ee-7c94-4665-9663-ef60af596098 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.817465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:16.855905 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:16.874764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aaef9-c7d9-4d3d-b9c1-95f94911d5fc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.880747 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e85ee-7c94-4665-9663-ef60af596098 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:16.906580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f20f-6279-4461-842d-87ef1673ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.907202 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f20f-6279-4461-842d-87ef1673ef68 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.907480 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f20f-6279-4461-842d-87ef1673ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.907748 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f20f-6279-4461-842d-87ef1673ef68 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.908385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f20f-6279-4461-842d-87ef1673ef68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:16.908684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f20f-6279-4461-842d-87ef1673ef68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.910064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f20f-6279-4461-842d-87ef1673ef68 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.913129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49487cb-01a8-4813-901f-7707f8491936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:16.913726 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49487cb-01a8-4813-901f-7707f8491936 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:16.914002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49487cb-01a8-4813-901f-7707f8491936 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:16.914272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49487cb-01a8-4813-901f-7707f8491936 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:16.914961 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49487cb-01a8-4813-901f-7707f8491936 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:16.915254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49487cb-01a8-4813-901f-7707f8491936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:16.916615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49487cb-01a8-4813-901f-7707f8491936 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:16.921578 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 228/452] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:16.922556 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f20f-6279-4461-842d-87ef1673ef68 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:16.925467 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 225/453] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:16.926493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49487cb-01a8-4813-901f-7707f8491936 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.016512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.022433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49487cb-01a8-4813-901f-7707f8491936 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.025576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f20f-6279-4461-842d-87ef1673ef68 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.056257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ff9e0-f91c-40eb-8940-4184cbd6d36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.056881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ff9e0-f91c-40eb-8940-4184cbd6d36f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.057155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ff9e0-f91c-40eb-8940-4184cbd6d36f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.057411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ff9e0-f91c-40eb-8940-4184cbd6d36f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.058735 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f38d88-55ac-433b-bffd-0f7d46b6385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.059331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f38d88-55ac-433b-bffd-0f7d46b6385a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.059603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f38d88-55ac-433b-bffd-0f7d46b6385a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.060286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f38d88-55ac-433b-bffd-0f7d46b6385a None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.060498 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f38d88-55ac-433b-bffd-0f7d46b6385a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:17.060794 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f38d88-55ac-433b-bffd-0f7d46b6385a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.062155 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f38d88-55ac-433b-bffd-0f7d46b6385a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.064706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ff9e0-f91c-40eb-8940-4184cbd6d36f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.066124 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ff9e0-f91c-40eb-8940-4184cbd6d36f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.070942 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.074760 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 229/454] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-cc6ff9e0-f91c-40eb-8940-4184cbd6d36f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:17.077457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:17.078376 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 08:35:17.079773 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:17.080286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f38d88-55ac-433b-bffd-0f7d46b6385a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.080537 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 226/455] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:17.081569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ff9e0-f91c-40eb-8940-4184cbd6d36f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.178542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f38d88-55ac-433b-bffd-0f7d46b6385a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.178771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ff9e0-f91c-40eb-8940-4184cbd6d36f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.211225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7609da-4ea3-432a-b61c-f5aae623f71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.211842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7609da-4ea3-432a-b61c-f5aae623f71f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.212115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7609da-4ea3-432a-b61c-f5aae623f71f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.212381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7609da-4ea3-432a-b61c-f5aae623f71f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.212494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea5971-5749-4e7c-b803-ceb23e7128ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.213096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea5971-5749-4e7c-b803-ceb23e7128ee None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.213374 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea5971-5749-4e7c-b803-ceb23e7128ee None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.213651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea5971-5749-4e7c-b803-ceb23e7128ee None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.214309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea5971-5749-4e7c-b803-ceb23e7128ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:17.214627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea5971-5749-4e7c-b803-ceb23e7128ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.216019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea5971-5749-4e7c-b803-ceb23e7128ee None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.218956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7609da-4ea3-432a-b61c-f5aae623f71f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.220383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7609da-4ea3-432a-b61c-f5aae623f71f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.227210 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 230/456] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:17.227882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea5971-5749-4e7c-b803-ceb23e7128ee None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2d7609da-4ea3-432a-b61c-f5aae623f71f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:17.231243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:17.232243 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 227/457] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:17.233079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7609da-4ea3-432a-b61c-f5aae623f71f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.255318 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.287824 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.327933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea5971-5749-4e7c-b803-ceb23e7128ee None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.338293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7609da-4ea3-432a-b61c-f5aae623f71f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.359704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbb137-2450-4d72-b6e0-f20d720f5477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.360320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbb137-2450-4d72-b6e0-f20d720f5477 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.360592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbb137-2450-4d72-b6e0-f20d720f5477 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.360859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbb137-2450-4d72-b6e0-f20d720f5477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.371002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbb137-2450-4d72-b6e0-f20d720f5477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.371253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828edb14-ee01-4f93-925f-82b2a34981dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.371881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828edb14-ee01-4f93-925f-82b2a34981dc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.372156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828edb14-ee01-4f93-925f-82b2a34981dc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.372422 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828edb14-ee01-4f93-925f-82b2a34981dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.372513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbb137-2450-4d72-b6e0-f20d720f5477 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.379624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828edb14-ee01-4f93-925f-82b2a34981dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.381039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828edb14-ee01-4f93-925f-82b2a34981dc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-20dbb137-2450-4d72-b6e0-f20d720f5477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:17.386121 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:17.387110 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:17.387971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:17.387971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:17.387971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:17.387971 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 231/458] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:17.388185 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbb137-2450-4d72-b6e0-f20d720f5477 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-828edb14-ee01-4f93-925f-82b2a34981dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:17.392098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:17.393010 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 228/459] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:17.393895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828edb14-ee01-4f93-925f-82b2a34981dc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.468230 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.487551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbb137-2450-4d72-b6e0-f20d720f5477 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.490657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828edb14-ee01-4f93-925f-82b2a34981dc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.501166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.522590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43832796-c772-40bc-8444-59227993ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.523019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93225c6-d886-4e68-a74e-fe74360efa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.523214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43832796-c772-40bc-8444-59227993ef80 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.523484 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43832796-c772-40bc-8444-59227993ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.523639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93225c6-d886-4e68-a74e-fe74360efa40 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.523733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43832796-c772-40bc-8444-59227993ef80 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.523916 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93225c6-d886-4e68-a74e-fe74360efa40 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.524184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93225c6-d886-4e68-a74e-fe74360efa40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.524452 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43832796-c772-40bc-8444-59227993ef80 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:17.524751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43832796-c772-40bc-8444-59227993ef80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.526186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43832796-c772-40bc-8444-59227993ef80 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.530667 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93225c6-d886-4e68-a74e-fe74360efa40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.532156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93225c6-d886-4e68-a74e-fe74360efa40 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.543402 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 232/460] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e93225c6-d886-4e68-a74e-fe74360efa40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:17.543603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:17.544427 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:17.545488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:17.545488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:17.545488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:17.545488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:17.545488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:17.545488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:17.545488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:17.545488 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:17.546004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43832796-c772-40bc-8444-59227993ef80 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.546487 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 229/461] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:17.547330 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93225c6-d886-4e68-a74e-fe74360efa40 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.642033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93225c6-d886-4e68-a74e-fe74360efa40 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.643968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43832796-c772-40bc-8444-59227993ef80 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.675172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302eb162-c634-40e4-ae58-d9cca479234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.675783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302eb162-c634-40e4-ae58-d9cca479234d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.676062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302eb162-c634-40e4-ae58-d9cca479234d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.676339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302eb162-c634-40e4-ae58-d9cca479234d None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.676922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e322068-4f79-4e4e-8bb4-5be78525c6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.677032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302eb162-c634-40e4-ae58-d9cca479234d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:17.677262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302eb162-c634-40e4-ae58-d9cca479234d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.677513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e322068-4f79-4e4e-8bb4-5be78525c6fb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.677788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e322068-4f79-4e4e-8bb4-5be78525c6fb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.678053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e322068-4f79-4e4e-8bb4-5be78525c6fb None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.678635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302eb162-c634-40e4-ae58-d9cca479234d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.678752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e322068-4f79-4e4e-8bb4-5be78525c6fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:17.679045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e322068-4f79-4e4e-8bb4-5be78525c6fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.680449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e322068-4f79-4e4e-8bb4-5be78525c6fb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.680879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.690710 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 230/462] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:17.696386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302eb162-c634-40e4-ae58-d9cca479234d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.696516 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 233/463] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:17.696516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e322068-4f79-4e4e-8bb4-5be78525c6fb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.723857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.790645 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302eb162-c634-40e4-ae58-d9cca479234d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.791474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e322068-4f79-4e4e-8bb4-5be78525c6fb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.822842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05122027-3934-4a2f-ab91-ca315f9440c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.823416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5eca-d5b9-436f-b283-bcde9b6be88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.823568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05122027-3934-4a2f-ab91-ca315f9440c8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.823680 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05122027-3934-4a2f-ab91-ca315f9440c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.823981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05122027-3934-4a2f-ab91-ca315f9440c8 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.824108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5eca-d5b9-436f-b283-bcde9b6be88e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.824307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5eca-d5b9-436f-b283-bcde9b6be88e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.824576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5eca-d5b9-436f-b283-bcde9b6be88e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.824702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05122027-3934-4a2f-ab91-ca315f9440c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:17.824978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05122027-3934-4a2f-ab91-ca315f9440c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.826399 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05122027-3934-4a2f-ab91-ca315f9440c8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.842915 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 231/464] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:17.843692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05122027-3934-4a2f-ab91-ca315f9440c8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.890604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5eca-d5b9-436f-b283-bcde9b6be88e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.892158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5eca-d5b9-436f-b283-bcde9b6be88e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.897004 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.923444 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:17.939611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05122027-3934-4a2f-ab91-ca315f9440c8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:17.975210 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fa92a-583a-496b-ae72-68bda732b3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:17.975826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fa92a-583a-496b-ae72-68bda732b3f6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:17.976101 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fa92a-583a-496b-ae72-68bda732b3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:17.976371 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fa92a-583a-496b-ae72-68bda732b3f6 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:17.977058 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fa92a-583a-496b-ae72-68bda732b3f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:17.977360 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fa92a-583a-496b-ae72-68bda732b3f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:17.978843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fa92a-583a-496b-ae72-68bda732b3f6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:17.990126 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 232/465] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:17.991021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fa92a-583a-496b-ae72-68bda732b3f6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:17.993731 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 234/466] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:35:17.994474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5eca-d5b9-436f-b283-bcde9b6be88e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.086394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fa92a-583a-496b-ae72-68bda732b3f6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:18.094296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5eca-d5b9-436f-b283-bcde9b6be88e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:18.099511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:18.119137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68867bff-7bb2-4db5-b512-3fd6b9d1467e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:18.119774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68867bff-7bb2-4db5-b512-3fd6b9d1467e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:18.120054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68867bff-7bb2-4db5-b512-3fd6b9d1467e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:18.120325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68867bff-7bb2-4db5-b512-3fd6b9d1467e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:18.122739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:18.127901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68867bff-7bb2-4db5-b512-3fd6b9d1467e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:18.129081 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765d9c1-4650-445a-9bb1-d571131a44c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:18.129332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68867bff-7bb2-4db5-b512-3fd6b9d1467e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:18.129735 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765d9c1-4650-445a-9bb1-d571131a44c9 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:18.130012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765d9c1-4650-445a-9bb1-d571131a44c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:18.130284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765d9c1-4650-445a-9bb1-d571131a44c9 None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/dbf288ceecac4fec8b2161c55a61a8ba/roles/cb7d27022ded4ea2afd3d5ebaefbf429` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-68867bff-7bb2-4db5-b512-3fd6b9d1467e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:18.140098 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:18.140995 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 233/467] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:18.141922 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68867bff-7bb2-4db5-b512-3fd6b9d1467e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.149100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765d9c1-4650-445a-9bb1-d571131a44c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=dbf288ceecac4fec8b2161c55a61a8ba, role_id=cb7d27022ded4ea2afd3d5ebaefbf429)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:18.150555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6765d9c1-4650-445a-9bb1-d571131a44c9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:18.164688 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 235/468] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:35:17 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/dbf288ceecac4fec8b2161c55a61a8ba/roles/cb7d27022ded4ea2afd3d5ebaefbf429 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:35:18.165602 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765d9c1-4650-445a-9bb1-d571131a44c9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.269369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68867bff-7bb2-4db5-b512-3fd6b9d1467e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:18.276711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765d9c1-4650-445a-9bb1-d571131a44c9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:18.309279 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998874c9-1514-4577-b64f-a881ee2dae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:18.309910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998874c9-1514-4577-b64f-a881ee2dae11 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:18.310185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998874c9-1514-4577-b64f-a881ee2dae11 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:18.310451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998874c9-1514-4577-b64f-a881ee2dae11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:18.320596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9836b8-fe5c-468a-8dcb-9d2a8b78e873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:18.321670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9836b8-fe5c-468a-8dcb-9d2a8b78e873 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:18.321954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9836b8-fe5c-468a-8dcb-9d2a8b78e873 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:18.322226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9836b8-fe5c-468a-8dcb-9d2a8b78e873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:18.323104 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9836b8-fe5c-468a-8dcb-9d2a8b78e873 None admin] RBAC: Adding query filter params (role.id=cb7d27022ded4ea2afd3d5ebaefbf429, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=f257d9b665714bff8b4ceb4fc5dc8387) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:18.323416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9836b8-fe5c-468a-8dcb-9d2a8b78e873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:18.324940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9836b8-fe5c-468a-8dcb-9d2a8b78e873 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:18.338746 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 236/469] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:35:18 2022] GET /identity/v3/role_assignments?user.id=f257d9b665714bff8b4ceb4fc5dc8387&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=cb7d27022ded4ea2afd3d5ebaefbf429 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:18.338746 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9836b8-fe5c-468a-8dcb-9d2a8b78e873 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.374063 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:18.403640 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:18.414853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998874c9-1514-4577-b64f-a881ee2dae11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:18.417421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998874c9-1514-4577-b64f-a881ee2dae11 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:18.480231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9836b8-fe5c-468a-8dcb-9d2a8b78e873 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:18.525489 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08be0-1c15-46a0-993c-4f9106a3654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:18.526128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08be0-1c15-46a0-993c-4f9106a3654c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:18.526401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08be0-1c15-46a0-993c-4f9106a3654c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:18.526671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08be0-1c15-46a0-993c-4f9106a3654c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:18.540821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce08be0-1c15-46a0-993c-4f9106a3654c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:18.542471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce08be0-1c15-46a0-993c-4f9106a3654c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-7ce08be0-1c15-46a0-993c-4f9106a3654c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:18.560351 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:18.561304 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:18.562575 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:18.562575 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:18.562575 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:18.562575 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 237/470] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:18.562575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08be0-1c15-46a0-993c-4f9106a3654c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.567755 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 234/471] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:35:18.571746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998874c9-1514-4577-b64f-a881ee2dae11 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.649164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:18.676064 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998874c9-1514-4577-b64f-a881ee2dae11 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:18.690620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:18.698924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08be0-1c15-46a0-993c-4f9106a3654c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:18.715261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fc40c-3eac-4a4b-9ed0-0a33a6821eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:18.715975 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fc40c-3eac-4a4b-9ed0-0a33a6821eaa None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:18.716250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fc40c-3eac-4a4b-9ed0-0a33a6821eaa None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:18.716516 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fc40c-3eac-4a4b-9ed0-0a33a6821eaa None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:18.717415 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fc40c-3eac-4a4b-9ed0-0a33a6821eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:18.717549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fc40c-3eac-4a4b-9ed0-0a33a6821eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:18.718995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fc40c-3eac-4a4b-9ed0-0a33a6821eaa None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:18.731617 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 235/472] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:18.732771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fc40c-3eac-4a4b-9ed0-0a33a6821eaa None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.741570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150fb60-6b5c-4ca5-85e1-bb79d5a5a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:18.743047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150fb60-6b5c-4ca5-85e1-bb79d5a5a369 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:18.743520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150fb60-6b5c-4ca5-85e1-bb79d5a5a369 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:18.744403 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150fb60-6b5c-4ca5-85e1-bb79d5a5a369 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:18.745321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150fb60-6b5c-4ca5-85e1-bb79d5a5a369 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:18.746204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150fb60-6b5c-4ca5-85e1-bb79d5a5a369 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:18.748423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150fb60-6b5c-4ca5-85e1-bb79d5a5a369 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:18.777106 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 238/473] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Oct 24 08:35:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:18.777106 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150fb60-6b5c-4ca5-85e1-bb79d5a5a369 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.861486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fc40c-3eac-4a4b-9ed0-0a33a6821eaa None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:18.881798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150fb60-6b5c-4ca5-85e1-bb79d5a5a369 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:18.911003 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced61101-dc58-4704-8b46-4d16d75970e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:18.912543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced61101-dc58-4704-8b46-4d16d75970e1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:18.913327 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced61101-dc58-4704-8b46-4d16d75970e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:18.914024 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced61101-dc58-4704-8b46-4d16d75970e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:18.916137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:18.918295 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e685d70-6b45-47e9-9d3e-3e7d42739a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:18.919080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e685d70-6b45-47e9-9d3e-3e7d42739a67 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:18.919428 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e685d70-6b45-47e9-9d3e-3e7d42739a67 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:18.919915 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e685d70-6b45-47e9-9d3e-3e7d42739a67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:18.927979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e685d70-6b45-47e9-9d3e-3e7d42739a67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:18.928504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced61101-dc58-4704-8b46-4d16d75970e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:18.929664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e685d70-6b45-47e9-9d3e-3e7d42739a67 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:18.930337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced61101-dc58-4704-8b46-4d16d75970e1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9e685d70-6b45-47e9-9d3e-3e7d42739a67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:18.942080 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:18.943019 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:18.943851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:18.943851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:18.943851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:18.943851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:18.943851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:18.943851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:18.943851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:18.943851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:18.943851 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 239/474] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:18.944902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e685d70-6b45-47e9-9d3e-3e7d42739a67 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ced61101-dc58-4704-8b46-4d16d75970e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:18.950280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:18.951194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:18.952047 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:18.952047 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:18.952047 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:18.954524 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 236/475] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:18.955590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced61101-dc58-4704-8b46-4d16d75970e1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:18.967166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:19.069689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e685d70-6b45-47e9-9d3e-3e7d42739a67 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:19.074601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced61101-dc58-4704-8b46-4d16d75970e1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:19.112527 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999de893-83ac-4875-b2fa-aae603b48f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:19.113168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999de893-83ac-4875-b2fa-aae603b48f6f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:19.113525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999de893-83ac-4875-b2fa-aae603b48f6f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:19.113695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999de893-83ac-4875-b2fa-aae603b48f6f None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:19.114373 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999de893-83ac-4875-b2fa-aae603b48f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:19.114663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999de893-83ac-4875-b2fa-aae603b48f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:19.116119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999de893-83ac-4875-b2fa-aae603b48f6f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:19.127808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702b6f2-e1ee-4f72-990b-32ecffb79b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:19.128589 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 237/476] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:19.129222 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702b6f2-e1ee-4f72-990b-32ecffb79b7d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:19.129640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999de893-83ac-4875-b2fa-aae603b48f6f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:19.129910 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702b6f2-e1ee-4f72-990b-32ecffb79b7d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:19.130455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702b6f2-e1ee-4f72-990b-32ecffb79b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:19.176905 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:19.220671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702b6f2-e1ee-4f72-990b-32ecffb79b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:19.222072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702b6f2-e1ee-4f72-990b-32ecffb79b7d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:19.237489 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:19.242199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999de893-83ac-4875-b2fa-aae603b48f6f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:19.285797 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7375dba-7786-4dd7-8839-21ae41e12b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:19.286943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7375dba-7786-4dd7-8839-21ae41e12b92 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:19.287317 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7375dba-7786-4dd7-8839-21ae41e12b92 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:19.287680 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7375dba-7786-4dd7-8839-21ae41e12b92 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:19.288439 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7375dba-7786-4dd7-8839-21ae41e12b92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:19.288836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7375dba-7786-4dd7-8839-21ae41e12b92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:19.291308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7375dba-7786-4dd7-8839-21ae41e12b92 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:19.308569 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 238/477] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:19.313072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7375dba-7786-4dd7-8839-21ae41e12b92 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:19.358802 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 240/478] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:35:19.359974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702b6f2-e1ee-4f72-990b-32ecffb79b7d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:19.422377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7375dba-7786-4dd7-8839-21ae41e12b92 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:19.429449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:19.455894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be458c9-d630-480f-9ae7-783788ec639e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:19.456613 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be458c9-d630-480f-9ae7-783788ec639e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:19.456950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be458c9-d630-480f-9ae7-783788ec639e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:19.457285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be458c9-d630-480f-9ae7-783788ec639e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:19.458069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be458c9-d630-480f-9ae7-783788ec639e None admin] RBAC: Adding query filter params (role.id=cb7d27022ded4ea2afd3d5ebaefbf429, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=6046d6440af84aa28aab47a018638e18) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:19.458435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be458c9-d630-480f-9ae7-783788ec639e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:19.459980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be458c9-d630-480f-9ae7-783788ec639e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:19.466204 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 239/479] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:35:19 2022] GET /identity/v3/role_assignments?user.id=6046d6440af84aa28aab47a018638e18&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=cb7d27022ded4ea2afd3d5ebaefbf429 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:19.467173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be458c9-d630-480f-9ae7-783788ec639e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:19.471325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702b6f2-e1ee-4f72-990b-32ecffb79b7d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:19.488375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:19.512573 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4079f13-043b-46f3-982d-f21672e49216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:19.513242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4079f13-043b-46f3-982d-f21672e49216 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:19.513522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4079f13-043b-46f3-982d-f21672e49216 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:19.513789 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4079f13-043b-46f3-982d-f21672e49216 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:19.521632 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4079f13-043b-46f3-982d-f21672e49216 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:19.523181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4079f13-043b-46f3-982d-f21672e49216 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d4079f13-043b-46f3-982d-f21672e49216 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:19.535543 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:19.536825 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:19.537845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:19.537845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:19.537845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:19.537845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:19.537845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:19.537845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:19.537845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:19.537845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:19.537845 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 241/480] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:19.543186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4079f13-043b-46f3-982d-f21672e49216 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:19.654314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4079f13-043b-46f3-982d-f21672e49216 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:19.694872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be458c9-d630-480f-9ae7-783788ec639e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:19.718128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e9146-1831-423a-84db-4c3d3c98c53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:19.718917 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e9146-1831-423a-84db-4c3d3c98c53d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:19.719194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e9146-1831-423a-84db-4c3d3c98c53d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:19.719462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e9146-1831-423a-84db-4c3d3c98c53d None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:19.720161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e9146-1831-423a-84db-4c3d3c98c53d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:19.720467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e9146-1831-423a-84db-4c3d3c98c53d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:19.722213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e9146-1831-423a-84db-4c3d3c98c53d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:19.731990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207a5f6-fb84-41c3-a608-1e204894377b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:19.732627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207a5f6-fb84-41c3-a608-1e204894377b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:19.732903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207a5f6-fb84-41c3-a608-1e204894377b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:19.733171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207a5f6-fb84-41c3-a608-1e204894377b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:19.735339 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 242/481] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:19.740832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e9146-1831-423a-84db-4c3d3c98c53d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:19.744432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207a5f6-fb84-41c3-a608-1e204894377b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:19.745938 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207a5f6-fb84-41c3-a608-1e204894377b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f207a5f6-fb84-41c3-a608-1e204894377b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:19.760078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:19.761043 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:19.762779 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:19.762779 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:19.762779 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:19.762779 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 240/482] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:19.767031 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207a5f6-fb84-41c3-a608-1e204894377b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:19.793136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:19.795046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:19.857194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e9146-1831-423a-84db-4c3d3c98c53d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:19.872635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207a5f6-fb84-41c3-a608-1e204894377b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:19.900499 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8738bd0-7b1f-44a0-bb6f-cff405551d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:19.901651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8738bd0-7b1f-44a0-bb6f-cff405551d00 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:19.902164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8738bd0-7b1f-44a0-bb6f-cff405551d00 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:19.902634 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8738bd0-7b1f-44a0-bb6f-cff405551d00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:19.903727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8738bd0-7b1f-44a0-bb6f-cff405551d00 None admin] RBAC: Adding query filter params (role.id=cb7d27022ded4ea2afd3d5ebaefbf429, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=3b1c2ea68e4741728469ab7cc542b6d5) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:19.904202 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8738bd0-7b1f-44a0-bb6f-cff405551d00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:19.905800 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8738bd0-7b1f-44a0-bb6f-cff405551d00 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:19.912976 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 243/483] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:35:19 2022] GET /identity/v3/role_assignments?user.id=3b1c2ea68e4741728469ab7cc542b6d5&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=cb7d27022ded4ea2afd3d5ebaefbf429 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:19.913967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d7e12-6f75-4462-91dd-5066e4c0de94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:19.914585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d7e12-6f75-4462-91dd-5066e4c0de94 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:19.914891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d7e12-6f75-4462-91dd-5066e4c0de94 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:19.915165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d7e12-6f75-4462-91dd-5066e4c0de94 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:19.915810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d7e12-6f75-4462-91dd-5066e4c0de94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:19.916110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d7e12-6f75-4462-91dd-5066e4c0de94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:19.917623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d7e12-6f75-4462-91dd-5066e4c0de94 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:19.935167 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 241/484] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:19.940803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8738bd0-7b1f-44a0-bb6f-cff405551d00 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:20.044547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:20.058092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8738bd0-7b1f-44a0-bb6f-cff405551d00 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:20.106561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164212f-071c-41e8-b3c3-f07f7256e28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:20.107604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164212f-071c-41e8-b3c3-f07f7256e28a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:20.108054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164212f-071c-41e8-b3c3-f07f7256e28a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:20.108495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164212f-071c-41e8-b3c3-f07f7256e28a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:20.122316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164212f-071c-41e8-b3c3-f07f7256e28a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:20.124078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164212f-071c-41e8-b3c3-f07f7256e28a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3164212f-071c-41e8-b3c3-f07f7256e28a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:20.143125 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:20.144563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:20.145407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:20.145407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:20.145407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:20.145407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:20.145407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:20.145407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:20.145407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:20.145407 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:20.147173 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 244/485] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:20.155956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d7e12-6f75-4462-91dd-5066e4c0de94 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:20.279427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:20.302677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d7e12-6f75-4462-91dd-5066e4c0de94 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:20.340917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af1e24-008d-4fba-8f7f-678ed6592758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:20.341569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af1e24-008d-4fba-8f7f-678ed6592758 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:20.341850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af1e24-008d-4fba-8f7f-678ed6592758 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:20.342126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af1e24-008d-4fba-8f7f-678ed6592758 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:20.342896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af1e24-008d-4fba-8f7f-678ed6592758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:20.343383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af1e24-008d-4fba-8f7f-678ed6592758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:20.344889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af1e24-008d-4fba-8f7f-678ed6592758 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:20.359675 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 242/486] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:20.366423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164212f-071c-41e8-b3c3-f07f7256e28a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:20.480069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b45b9-5c8e-4a23-8924-973329832d69 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:20.480452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b45b9-5c8e-4a23-8924-973329832d69 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:20.480802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b45b9-5c8e-4a23-8924-973329832d69 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:20.482481 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 243/487] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:20.482866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164212f-071c-41e8-b3c3-f07f7256e28a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:20.487652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a20f0-24af-45ea-bc9c-1c06d7f4d399 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:20.488005 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a20f0-24af-45ea-bc9c-1c06d7f4d399 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:20.488354 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a20f0-24af-45ea-bc9c-1c06d7f4d399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:20.509291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-061a20f0-24af-45ea-bc9c-1c06d7f4d399 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:20.510612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-061a20f0-24af-45ea-bc9c-1c06d7f4d399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:20.525166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d3178-cb60-4119-b198-707d5b3d7244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:20.525845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d3178-cb60-4119-b198-707d5b3d7244 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:20.526214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d3178-cb60-4119-b198-707d5b3d7244 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:20.526492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d3178-cb60-4119-b198-707d5b3d7244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:20.527369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d3178-cb60-4119-b198-707d5b3d7244 None admin] RBAC: Adding query filter params (role.id=cb7d27022ded4ea2afd3d5ebaefbf429, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=2254623f523c48f0aa454b85be41df95) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:20.527777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d3178-cb60-4119-b198-707d5b3d7244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:20.529213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d3178-cb60-4119-b198-707d5b3d7244 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:20.536519 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 245/488] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:35:20 2022] GET /identity/v3/role_assignments?user.id=2254623f523c48f0aa454b85be41df95&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=cb7d27022ded4ea2afd3d5ebaefbf429 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:20.630333 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 244/489] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 08:35:20.912561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dd804-9879-42d5-a68c-21fefa6010bb None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:20.913134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dd804-9879-42d5-a68c-21fefa6010bb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:20.913578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015dd804-9879-42d5-a68c-21fefa6010bb None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:20.915447 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 246/490] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:20.920855 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4025db3-ac70-4b3f-a81f-93649ccf8bbb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:20.921151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4025db3-ac70-4b3f-a81f-93649ccf8bbb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:20.921430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4025db3-ac70-4b3f-a81f-93649ccf8bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:20.943683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f4025db3-ac70-4b3f-a81f-93649ccf8bbb None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:20.945053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f4025db3-ac70-4b3f-a81f-93649ccf8bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:20.959753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015dd804-9879-42d5-a68c-21fefa6010bb None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:20.997052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:21.049383 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 245/491] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:21.131465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015dd804-9879-42d5-a68c-21fefa6010bb None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:21.169606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742de1c-f854-47a9-9692-ca1474736750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:21.170979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742de1c-f854-47a9-9692-ca1474736750 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:21.171443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742de1c-f854-47a9-9692-ca1474736750 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:21.171870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742de1c-f854-47a9-9692-ca1474736750 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:21.189792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742de1c-f854-47a9-9692-ca1474736750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-b742de1c-f854-47a9-9692-ca1474736750 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:21.195182 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:21.196093 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:21.196922 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 247/492] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:35:21.207868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4025db3-ac70-4b3f-a81f-93649ccf8bbb None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:21.210530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:21.335059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742de1c-f854-47a9-9692-ca1474736750 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:21.376155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4025db3-ac70-4b3f-a81f-93649ccf8bbb None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:21.444427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844d53a-2069-4ef1-8488-01765f0f8ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:21.445176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844d53a-2069-4ef1-8488-01765f0f8ac2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:21.445454 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844d53a-2069-4ef1-8488-01765f0f8ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:21.445723 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844d53a-2069-4ef1-8488-01765f0f8ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:21.447310 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844d53a-2069-4ef1-8488-01765f0f8ac2 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:21.447623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844d53a-2069-4ef1-8488-01765f0f8ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:21.451877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844d53a-2069-4ef1-8488-01765f0f8ac2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:21.466594 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 246/493] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:21.472760 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844d53a-2069-4ef1-8488-01765f0f8ac2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:21.509728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:21.544492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742de1c-f854-47a9-9692-ca1474736750 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:21.589698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:21.589924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513c925-e8e4-451f-9c97-1ffc481bee3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:21.590372 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513c925-e8e4-451f-9c97-1ffc481bee3d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:21.590651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513c925-e8e4-451f-9c97-1ffc481bee3d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:21.590944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513c925-e8e4-451f-9c97-1ffc481bee3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:21.616436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513c925-e8e4-451f-9c97-1ffc481bee3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d513c925-e8e4-451f-9c97-1ffc481bee3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:21.621926 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:21.623390 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:21.624377 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:21.625012 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 248/494] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:35:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:21.628874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513c925-e8e4-451f-9c97-1ffc481bee3d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:21.667752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844d53a-2069-4ef1-8488-01765f0f8ac2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:21.732978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6763158-1e29-4a4f-8f52-3d9b2288879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:21.733545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6763158-1e29-4a4f-8f52-3d9b2288879c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:21.733825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6763158-1e29-4a4f-8f52-3d9b2288879c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:21.734102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6763158-1e29-4a4f-8f52-3d9b2288879c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:21.753899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6763158-1e29-4a4f-8f52-3d9b2288879c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:21.755414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6763158-1e29-4a4f-8f52-3d9b2288879c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:21.758985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513c925-e8e4-451f-9c97-1ffc481bee3d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f6763158-1e29-4a4f-8f52-3d9b2288879c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:21.783569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:21.784566 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:21.785443 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:21.785443 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:21.785443 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:21.785443 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 247/495] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:21.792580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6763158-1e29-4a4f-8f52-3d9b2288879c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:21.831319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9705d-bf24-4460-90b9-1d34374fe740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:21.831946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9705d-bf24-4460-90b9-1d34374fe740 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:21.832225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9705d-bf24-4460-90b9-1d34374fe740 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:21.832498 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9705d-bf24-4460-90b9-1d34374fe740 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:21.833167 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9705d-bf24-4460-90b9-1d34374fe740 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:21.833470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9705d-bf24-4460-90b9-1d34374fe740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:21.834902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9705d-bf24-4460-90b9-1d34374fe740 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:21.850540 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 249/496] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:21.856237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9705d-bf24-4460-90b9-1d34374fe740 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:21.863086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:21.992558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6763158-1e29-4a4f-8f52-3d9b2288879c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:22.001486 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9705d-bf24-4460-90b9-1d34374fe740 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:22.022633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:22.044435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8cb88-0e93-4aec-9732-6f06295a8426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:22.045075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8cb88-0e93-4aec-9732-6f06295a8426 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.045351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8cb88-0e93-4aec-9732-6f06295a8426 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.045617 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8cb88-0e93-4aec-9732-6f06295a8426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.059365 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a674b68-acc6-4fe9-9567-3dc4b7a0ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:22.059984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a674b68-acc6-4fe9-9567-3dc4b7a0ad02 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.060257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a674b68-acc6-4fe9-9567-3dc4b7a0ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.060526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a674b68-acc6-4fe9-9567-3dc4b7a0ad02 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.061168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a674b68-acc6-4fe9-9567-3dc4b7a0ad02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:22.061468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a674b68-acc6-4fe9-9567-3dc4b7a0ad02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:22.062876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a674b68-acc6-4fe9-9567-3dc4b7a0ad02 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:22.069510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8cb88-0e93-4aec-9732-6f06295a8426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:22.070981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8cb88-0e93-4aec-9732-6f06295a8426 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:22.081157 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 248/497] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-08b8cb88-0e93-4aec-9732-6f06295a8426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:22.090754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:22.091709 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:22.092671 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:22.092671 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:22.092671 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:22.092671 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 250/498] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:35:22.092900 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a674b68-acc6-4fe9-9567-3dc4b7a0ad02 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:22.092980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8cb88-0e93-4aec-9732-6f06295a8426 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:22.155082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:22.221501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8cb88-0e93-4aec-9732-6f06295a8426 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:22.271621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1cc16-1a72-49d7-a59f-23e826328342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:22.272574 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1cc16-1a72-49d7-a59f-23e826328342 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.273069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1cc16-1a72-49d7-a59f-23e826328342 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.273537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1cc16-1a72-49d7-a59f-23e826328342 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.274390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1cc16-1a72-49d7-a59f-23e826328342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:22.274918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1cc16-1a72-49d7-a59f-23e826328342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:22.276533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1cc16-1a72-49d7-a59f-23e826328342 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:22.295750 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 251/499] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:22.302773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1cc16-1a72-49d7-a59f-23e826328342 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:22.320641 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a674b68-acc6-4fe9-9567-3dc4b7a0ad02 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:22.377891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bdc59-961c-41eb-b02b-4f19c6655794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:22.378692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bdc59-961c-41eb-b02b-4f19c6655794 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.379083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bdc59-961c-41eb-b02b-4f19c6655794 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.379451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bdc59-961c-41eb-b02b-4f19c6655794 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.387919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bdc59-961c-41eb-b02b-4f19c6655794 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:22.389479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bdc59-961c-41eb-b02b-4f19c6655794 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f66bdc59-961c-41eb-b02b-4f19c6655794 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:22.401100 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:22.402283 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:22.403790 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 249/500] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:35:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:22.409644 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd913-f47f-43aa-9494-71ba45f251f9 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.409989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd913-f47f-43aa-9494-71ba45f251f9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.410334 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd913-f47f-43aa-9494-71ba45f251f9 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.412324 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 250/501] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:22.413087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd913-f47f-43aa-9494-71ba45f251f9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:22.422067 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1cc16-1a72-49d7-a59f-23e826328342 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:22.444289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:22.452753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:22.470844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dee772-3679-4716-8ec9-e83c98fd901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:22.471480 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dee772-3679-4716-8ec9-e83c98fd901c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.472260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dee772-3679-4716-8ec9-e83c98fd901c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.472546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dee772-3679-4716-8ec9-e83c98fd901c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.482832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dee772-3679-4716-8ec9-e83c98fd901c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:22.484286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dee772-3679-4716-8ec9-e83c98fd901c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-24dee772-3679-4716-8ec9-e83c98fd901c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:22.499783 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:22.501620 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 252/502] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Oct 24 08:35:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e7bf5-ed89-4b89-b28e-99a811cc8881 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.503246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e7bf5-ed89-4b89-b28e-99a811cc8881 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.504485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e7bf5-ed89-4b89-b28e-99a811cc8881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.527196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd913-f47f-43aa-9494-71ba45f251f9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:22.529343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f05e7bf5-ed89-4b89-b28e-99a811cc8881 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:22.531486 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f05e7bf5-ed89-4b89-b28e-99a811cc8881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:22.561983 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280015a-0096-4cc5-8833-b1b8bf286c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:22.562683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280015a-0096-4cc5-8833-b1b8bf286c59 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.563053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280015a-0096-4cc5-8833-b1b8bf286c59 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.563323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280015a-0096-4cc5-8833-b1b8bf286c59 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.564207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280015a-0096-4cc5-8833-b1b8bf286c59 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:22.564514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280015a-0096-4cc5-8833-b1b8bf286c59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:22.565939 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280015a-0096-4cc5-8833-b1b8bf286c59 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:22.588394 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 251/503] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Oct 24 08:35:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:22.589345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280015a-0096-4cc5-8833-b1b8bf286c59 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:22.622827 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 253/504] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:22.624573 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e7bf5-ed89-4b89-b28e-99a811cc8881 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:22.728256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:22.738528 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280015a-0096-4cc5-8833-b1b8bf286c59 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:22.800207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4176b77-5033-4a01-b8f0-1bd86b649acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:22.800786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4176b77-5033-4a01-b8f0-1bd86b649acc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.801088 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4176b77-5033-4a01-b8f0-1bd86b649acc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.801308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4176b77-5033-4a01-b8f0-1bd86b649acc None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.801880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e7bf5-ed89-4b89-b28e-99a811cc8881 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:22.802038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4176b77-5033-4a01-b8f0-1bd86b649acc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:22.802340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4176b77-5033-4a01-b8f0-1bd86b649acc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:22.803812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4176b77-5033-4a01-b8f0-1bd86b649acc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:22.821399 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 252/505] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Oct 24 08:35:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:22.822642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7405c5-d330-45a4-9773-b2f6a1ebd63e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.825003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:22.827400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7405c5-d330-45a4-9773-b2f6a1ebd63e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.828068 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7405c5-d330-45a4-9773-b2f6a1ebd63e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.829564 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 253/506] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:22.834102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7405c5-d330-45a4-9773-b2f6a1ebd63e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:22.840612 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5e6b2-b3e1-471d-af05-e31e34ca9e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:22.841220 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5e6b2-b3e1-471d-af05-e31e34ca9e0f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.841496 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5e6b2-b3e1-471d-af05-e31e34ca9e0f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.841756 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5e6b2-b3e1-471d-af05-e31e34ca9e0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.910894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5e6b2-b3e1-471d-af05-e31e34ca9e0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:22.912700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5e6b2-b3e1-471d-af05-e31e34ca9e0f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:22.946390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7405c5-d330-45a4-9773-b2f6a1ebd63e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:22.985708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcef12-910f-4696-88d0-d55a1de1b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:22.986028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcef12-910f-4696-88d0-d55a1de1b5ff None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:22.986280 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcef12-910f-4696-88d0-d55a1de1b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:22.986549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcef12-910f-4696-88d0-d55a1de1b5ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:22.997052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcef12-910f-4696-88d0-d55a1de1b5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:22.999246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcef12-910f-4696-88d0-d55a1de1b5ff None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-dabcef12-910f-4696-88d0-d55a1de1b5ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:23.013968 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:23.014972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:23.015882 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:23.015882 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:23.015882 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:23.015882 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 254/507] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:23.016530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f378057-b860-4713-be28-84933b0ef055 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.016530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f378057-b860-4713-be28-84933b0ef055 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.016769 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f378057-b860-4713-be28-84933b0ef055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.023239 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 254/508] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 08:35:23.024296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5e6b2-b3e1-471d-af05-e31e34ca9e0f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.029085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:23.034799 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:23.041890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8f378057-b860-4713-be28-84933b0ef055 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:23.043176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8f378057-b860-4713-be28-84933b0ef055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:23.109699 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 255/509] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:23.110546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f378057-b860-4713-be28-84933b0ef055 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.151160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5e6b2-b3e1-471d-af05-e31e34ca9e0f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.194796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0876fd-f45a-464b-9de9-38d0103f3b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:23.195865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0876fd-f45a-464b-9de9-38d0103f3b2f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.196572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0876fd-f45a-464b-9de9-38d0103f3b2f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.197052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0876fd-f45a-464b-9de9-38d0103f3b2f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.198456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0876fd-f45a-464b-9de9-38d0103f3b2f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:23.200111 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0876fd-f45a-464b-9de9-38d0103f3b2f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-fb0876fd-f45a-464b-9de9-38d0103f3b2f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:23.214195 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:23.215314 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:23.222200 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 255/510] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:23.223044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0876fd-f45a-464b-9de9-38d0103f3b2f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.247237 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f378057-b860-4713-be28-84933b0ef055 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.294353 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c6d41-cf0e-4b84-81d4-1d3d4c3d1d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:23.295091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c6d41-cf0e-4b84-81d4-1d3d4c3d1d63 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.295367 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c6d41-cf0e-4b84-81d4-1d3d4c3d1d63 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.295654 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c6d41-cf0e-4b84-81d4-1d3d4c3d1d63 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.296426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c6d41-cf0e-4b84-81d4-1d3d4c3d1d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:23.296742 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c6d41-cf0e-4b84-81d4-1d3d4c3d1d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:23.298901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c6d41-cf0e-4b84-81d4-1d3d4c3d1d63 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.299970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:23.300511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:23.319535 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 256/511] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:23.319535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c6d41-cf0e-4b84-81d4-1d3d4c3d1d63 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.335857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0876fd-f45a-464b-9de9-38d0103f3b2f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.371424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd5d4d-978b-4176-805d-a1e3dd306a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:23.372055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd5d4d-978b-4176-805d-a1e3dd306a9b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.372333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd5d4d-978b-4176-805d-a1e3dd306a9b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.372604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd5d4d-978b-4176-805d-a1e3dd306a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.383489 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd5d4d-978b-4176-805d-a1e3dd306a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:23.384953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd5d4d-978b-4176-805d-a1e3dd306a9b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-52bd5d4d-978b-4176-805d-a1e3dd306a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:23.398534 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:23.399510 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:23.400404 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:23.400404 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:23.400404 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:23.400404 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 256/512] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:23.401054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09485675-6f2b-40c9-8d82-051da3b0603f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.401160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09485675-6f2b-40c9-8d82-051da3b0603f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.401682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09485675-6f2b-40c9-8d82-051da3b0603f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.404473 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 257/513] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:23.404473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09485675-6f2b-40c9-8d82-051da3b0603f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.422888 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c6d41-cf0e-4b84-81d4-1d3d4c3d1d63 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.455980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04856880-fa10-427b-88b1-c8a8455272fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:23.456595 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04856880-fa10-427b-88b1-c8a8455272fd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.456870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04856880-fa10-427b-88b1-c8a8455272fd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.457139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04856880-fa10-427b-88b1-c8a8455272fd None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.457759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04856880-fa10-427b-88b1-c8a8455272fd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:23.458083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04856880-fa10-427b-88b1-c8a8455272fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:23.459489 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04856880-fa10-427b-88b1-c8a8455272fd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.464064 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 257/514] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:23.464664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04856880-fa10-427b-88b1-c8a8455272fd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.507870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09485675-6f2b-40c9-8d82-051da3b0603f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.522117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:23.531315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:23.543431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32268eb-fa78-4a20-a2f3-7dbd7e1a0262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:23.544066 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32268eb-fa78-4a20-a2f3-7dbd7e1a0262 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.544344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32268eb-fa78-4a20-a2f3-7dbd7e1a0262 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.544616 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32268eb-fa78-4a20-a2f3-7dbd7e1a0262 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.551451 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32268eb-fa78-4a20-a2f3-7dbd7e1a0262 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:23.552878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32268eb-fa78-4a20-a2f3-7dbd7e1a0262 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e32268eb-fa78-4a20-a2f3-7dbd7e1a0262 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:23.563737 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:23.564772 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:23.565834 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:23.565834 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:23.565834 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:23.565834 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:23.565834 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:23.565834 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:23.565834 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:23.565834 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:23.566452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04856880-fa10-427b-88b1-c8a8455272fd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.566539 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 258/515] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:23.566539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32268eb-fa78-4a20-a2f3-7dbd7e1a0262 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.600333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaab758-8694-436b-93a2-38bb52eb1982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:23.600964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaab758-8694-436b-93a2-38bb52eb1982 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.601241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaab758-8694-436b-93a2-38bb52eb1982 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.601513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaab758-8694-436b-93a2-38bb52eb1982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.611491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaab758-8694-436b-93a2-38bb52eb1982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:23.612937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaab758-8694-436b-93a2-38bb52eb1982 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-8eaab758-8694-436b-93a2-38bb52eb1982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:23.626522 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:23.627481 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:23.628365 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:23.628365 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:23.628365 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:23.628365 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 258/516] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:23.628824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15e9b2-fd95-4034-b2f4-4d0d608309bf None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.629115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15e9b2-fd95-4034-b2f4-4d0d608309bf None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.629376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15e9b2-fd95-4034-b2f4-4d0d608309bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.646665 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ea15e9b2-fd95-4034-b2f4-4d0d608309bf None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:23.647893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ea15e9b2-fd95-4034-b2f4-4d0d608309bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:23.668850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32268eb-fa78-4a20-a2f3-7dbd7e1a0262 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.700808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d01fd-7c95-496d-bc25-8c2ca7b74543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:23.701405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d01fd-7c95-496d-bc25-8c2ca7b74543 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.701686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d01fd-7c95-496d-bc25-8c2ca7b74543 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.701957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d01fd-7c95-496d-bc25-8c2ca7b74543 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.702595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d01fd-7c95-496d-bc25-8c2ca7b74543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:23.702931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d01fd-7c95-496d-bc25-8c2ca7b74543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:23.704283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d01fd-7c95-496d-bc25-8c2ca7b74543 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.712934 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 259/517] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:23.713834 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15e9b2-fd95-4034-b2f4-4d0d608309bf None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.720231 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 259/518] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:23.720231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d01fd-7c95-496d-bc25-8c2ca7b74543 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.760534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:23.765748 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:23.819312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15e9b2-fd95-4034-b2f4-4d0d608309bf None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.832492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d01fd-7c95-496d-bc25-8c2ca7b74543 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.853165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab51df6-bf02-4a9a-b99e-036be5a27112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:23.853785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab51df6-bf02-4a9a-b99e-036be5a27112 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.854053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab51df6-bf02-4a9a-b99e-036be5a27112 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.854320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab51df6-bf02-4a9a-b99e-036be5a27112 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.855097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab51df6-bf02-4a9a-b99e-036be5a27112 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:23.855290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab51df6-bf02-4a9a-b99e-036be5a27112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:23.856706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab51df6-bf02-4a9a-b99e-036be5a27112 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.866902 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 260/519] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:23.867853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab51df6-bf02-4a9a-b99e-036be5a27112 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.880540 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29eb45-299e-422f-80f2-2aa462e77f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:23.881545 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29eb45-299e-422f-80f2-2aa462e77f13 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:23.882023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29eb45-299e-422f-80f2-2aa462e77f13 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:23.882490 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29eb45-299e-422f-80f2-2aa462e77f13 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:23.883448 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29eb45-299e-422f-80f2-2aa462e77f13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:23.883982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29eb45-299e-422f-80f2-2aa462e77f13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:23.885769 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29eb45-299e-422f-80f2-2aa462e77f13 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:23.899358 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 260/520] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:23.900663 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29eb45-299e-422f-80f2-2aa462e77f13 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:23.991846 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab51df6-bf02-4a9a-b99e-036be5a27112 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:23.994423 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:24.019219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29eb45-299e-422f-80f2-2aa462e77f13 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:24.021185 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:24.032047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44df97-3f2c-4c7c-acb5-385fd34392ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:24.032672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44df97-3f2c-4c7c-acb5-385fd34392ab None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:24.033188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44df97-3f2c-4c7c-acb5-385fd34392ab None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:24.033188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44df97-3f2c-4c7c-acb5-385fd34392ab None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:24.033843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44df97-3f2c-4c7c-acb5-385fd34392ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:24.034257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44df97-3f2c-4c7c-acb5-385fd34392ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:24.035598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44df97-3f2c-4c7c-acb5-385fd34392ab None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:24.053466 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 261/521] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:24.053466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44df97-3f2c-4c7c-acb5-385fd34392ab None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:24.066381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52de9d-95d4-4fb5-96f3-74b22fe0b5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:24.067069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52de9d-95d4-4fb5-96f3-74b22fe0b5c9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:24.067958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52de9d-95d4-4fb5-96f3-74b22fe0b5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:24.067958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52de9d-95d4-4fb5-96f3-74b22fe0b5c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:24.076491 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52de9d-95d4-4fb5-96f3-74b22fe0b5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:24.077969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52de9d-95d4-4fb5-96f3-74b22fe0b5c9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0f52de9d-95d4-4fb5-96f3-74b22fe0b5c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:24.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:24.090161 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:24.091092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:24.091092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:24.091092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:24.091092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:24.091092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:24.091092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:24.091092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:24.091092 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:24.091092 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 261/522] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:24.091587 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52de9d-95d4-4fb5-96f3-74b22fe0b5c9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:24.163295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44df97-3f2c-4c7c-acb5-385fd34392ab None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:24.196795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62ca53-543a-462a-97df-66da0986693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:24.197426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62ca53-543a-462a-97df-66da0986693c None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:24.197730 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62ca53-543a-462a-97df-66da0986693c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:24.198001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62ca53-543a-462a-97df-66da0986693c None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/f257d9b665714bff8b4ceb4fc5dc8387/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:24.199561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52de9d-95d4-4fb5-96f3-74b22fe0b5c9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:24.200163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62ca53-543a-462a-97df-66da0986693c None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=f257d9b665714bff8b4ceb4fc5dc8387, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:24.201617 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62ca53-543a-462a-97df-66da0986693c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:24.215752 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 262/523] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:35:24 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/f257d9b665714bff8b4ceb4fc5dc8387/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:35:24.216411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62ca53-543a-462a-97df-66da0986693c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:24.250034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfb7ba-adf6-48fa-8adf-84693fc7c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:24.250628 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:24.252285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfb7ba-adf6-48fa-8adf-84693fc7c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:24.252875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfb7ba-adf6-48fa-8adf-84693fc7c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:24.253346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfb7ba-adf6-48fa-8adf-84693fc7c8a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:24.254329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bfb7ba-adf6-48fa-8adf-84693fc7c8a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:24.256715 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bfb7ba-adf6-48fa-8adf-84693fc7c8a7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-22bfb7ba-adf6-48fa-8adf-84693fc7c8a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:24.268021 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:24.269124 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 08:35:24.269978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:24.274146 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 262/524] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:24 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:24.275448 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfb7ba-adf6-48fa-8adf-84693fc7c8a7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:24.314409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:24.370416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62ca53-543a-462a-97df-66da0986693c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:24.407137 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfb7ba-adf6-48fa-8adf-84693fc7c8a7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:24.410410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef027ab0-2480-496e-8985-b25eaadde225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:24.410410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef027ab0-2480-496e-8985-b25eaadde225 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:24.410690 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef027ab0-2480-496e-8985-b25eaadde225 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:24.411081 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef027ab0-2480-496e-8985-b25eaadde225 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:24.412172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef027ab0-2480-496e-8985-b25eaadde225 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:24.413656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef027ab0-2480-496e-8985-b25eaadde225 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:24.437220 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 263/525] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:35:24 2022] POST /identity/v3/users => generated 254 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:24.438302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef027ab0-2480-496e-8985-b25eaadde225 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:24.447019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604416e9-ed45-47a5-a8b8-33aac9c1f39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:24.447646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604416e9-ed45-47a5-a8b8-33aac9c1f39f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:24.447923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604416e9-ed45-47a5-a8b8-33aac9c1f39f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:24.449203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604416e9-ed45-47a5-a8b8-33aac9c1f39f None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:24.449203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604416e9-ed45-47a5-a8b8-33aac9c1f39f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:24.449492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604416e9-ed45-47a5-a8b8-33aac9c1f39f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:24.451108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604416e9-ed45-47a5-a8b8-33aac9c1f39f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:24.464905 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 263/526] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:24.471121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604416e9-ed45-47a5-a8b8-33aac9c1f39f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:24.564813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:24.589746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef027ab0-2480-496e-8985-b25eaadde225 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:24.629853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fc11b-c6ef-4ed3-b28b-46fb4ffdc4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:24.631053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fc11b-c6ef-4ed3-b28b-46fb4ffdc4c0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:24.631521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fc11b-c6ef-4ed3-b28b-46fb4ffdc4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:24.631987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fc11b-c6ef-4ed3-b28b-46fb4ffdc4c0 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:24.632880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fc11b-c6ef-4ed3-b28b-46fb4ffdc4c0 None admin] RBAC: Adding query filter params (name=network) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:24.633411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fc11b-c6ef-4ed3-b28b-46fb4ffdc4c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:24.635099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fc11b-c6ef-4ed3-b28b-46fb4ffdc4c0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:24.641962 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 264/527] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:24.648623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fc11b-c6ef-4ed3-b28b-46fb4ffdc4c0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:24.692826 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:24.735693 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604416e9-ed45-47a5-a8b8-33aac9c1f39f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:24.766934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fc11b-c6ef-4ed3-b28b-46fb4ffdc4c0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:24.783061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522935f-adc3-43ac-b669-5411a41f5422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:24.784217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522935f-adc3-43ac-b669-5411a41f5422 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:24.784677 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522935f-adc3-43ac-b669-5411a41f5422 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:24.785332 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522935f-adc3-43ac-b669-5411a41f5422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:24.787012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4522935f-adc3-43ac-b669-5411a41f5422 None admin] RBAC: Adding query filter params (role.id=cb7d27022ded4ea2afd3d5ebaefbf429, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=dbf288ceecac4fec8b2161c55a61a8ba) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:24.787526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4522935f-adc3-43ac-b669-5411a41f5422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:24.789685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4522935f-adc3-43ac-b669-5411a41f5422 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:24.799782 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 264/528] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:35:24 2022] GET /identity/v3/role_assignments?user.id=dbf288ceecac4fec8b2161c55a61a8ba&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=cb7d27022ded4ea2afd3d5ebaefbf429 => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:24.811823 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb8b8f-5205-40cf-9d99-2139ead3b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:24.812838 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb8b8f-5205-40cf-9d99-2139ead3b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:24.813303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb8b8f-5205-40cf-9d99-2139ead3b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:24.813747 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb8b8f-5205-40cf-9d99-2139ead3b9c3 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:24.814750 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb8b8f-5205-40cf-9d99-2139ead3b9c3 None admin] RBAC: Adding query filter params (type=network) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:24.815221 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb8b8f-5205-40cf-9d99-2139ead3b9c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:24.816904 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb8b8f-5205-40cf-9d99-2139ead3b9c3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:24.823404 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 265/529] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:26.163444 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13990f-db0d-426a-8965-0e6c6d9bd2d6 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:26.164019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13990f-db0d-426a-8965-0e6c6d9bd2d6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:26.164019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13990f-db0d-426a-8965-0e6c6d9bd2d6 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:26.165651 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 265/530] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:26.171178 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af36b1-e937-4f12-8e71-81614dc4d343 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:26.171573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af36b1-e937-4f12-8e71-81614dc4d343 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:26.171808 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af36b1-e937-4f12-8e71-81614dc4d343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:26.192715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-76af36b1-e937-4f12-8e71-81614dc4d343 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:26.193894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-76af36b1-e937-4f12-8e71-81614dc4d343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:26.196055 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:26.288471 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 266/531] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:26.644294 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13990f-db0d-426a-8965-0e6c6d9bd2d6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:26.768414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13990f-db0d-426a-8965-0e6c6d9bd2d6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:26.812511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a5697-491b-465e-ad3b-8b9853bc8e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:26.813139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a5697-491b-465e-ad3b-8b9853bc8e3b None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:26.813418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a5697-491b-465e-ad3b-8b9853bc8e3b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:26.813687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a5697-491b-465e-ad3b-8b9853bc8e3b None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:26.814348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a5697-491b-465e-ad3b-8b9853bc8e3b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:26.815772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a5697-491b-465e-ad3b-8b9853bc8e3b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:26.848074 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 266/532] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:35:26 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:26.957316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f59b-4442-47a7-a5b6-70fd82b78476 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:26.957502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f59b-4442-47a7-a5b6-70fd82b78476 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:26.957781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f59b-4442-47a7-a5b6-70fd82b78476 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:26.959553 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 267/533] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:26.965120 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f28a1b-bdc9-4d79-96c8-f011643fb1f6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:26.965408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f28a1b-bdc9-4d79-96c8-f011643fb1f6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:26.965687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f28a1b-bdc9-4d79-96c8-f011643fb1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:26.992236 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-33f28a1b-bdc9-4d79-96c8-f011643fb1f6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:26.993462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-33f28a1b-bdc9-4d79-96c8-f011643fb1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:26.998618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:27.020641 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ec4da-3050-45c9-a371-aa37cc29ed9c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.020970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ec4da-3050-45c9-a371-aa37cc29ed9c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.021197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ec4da-3050-45c9-a371-aa37cc29ed9c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.023618 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 268/534] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:27.043507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d15f7-a7c9-403b-bbe2-1f99f1e2e6d1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.043953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d15f7-a7c9-403b-bbe2-1f99f1e2e6d1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.044248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d15f7-a7c9-403b-bbe2-1f99f1e2e6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.078296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d89d15f7-a7c9-403b-bbe2-1f99f1e2e6d1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:27.079541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d89d15f7-a7c9-403b-bbe2-1f99f1e2e6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:27.114390 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 267/535] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:27.192971 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 269/536] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:27.436159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f28a1b-bdc9-4d79-96c8-f011643fb1f6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:27.543600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d15f7-a7c9-403b-bbe2-1f99f1e2e6d1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:27.544233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f28a1b-bdc9-4d79-96c8-f011643fb1f6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:27.587624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14541c5-bb51-458d-8a61-77aec548d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:27.588292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14541c5-bb51-458d-8a61-77aec548d33e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.588575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14541c5-bb51-458d-8a61-77aec548d33e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.588848 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14541c5-bb51-458d-8a61-77aec548d33e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.594345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:27.613584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14541c5-bb51-458d-8a61-77aec548d33e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f14541c5-bb51-458d-8a61-77aec548d33e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:27.628513 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:27.629717 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:27.630702 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 268/537] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:35:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:27.638055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14541c5-bb51-458d-8a61-77aec548d33e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:27.692528 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:27.736687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d15f7-a7c9-403b-bbe2-1f99f1e2e6d1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:27.772953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1c624-e858-4a0e-9648-a6333235ca61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:27.773897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1c624-e858-4a0e-9648-a6333235ca61 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.774359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1c624-e858-4a0e-9648-a6333235ca61 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.774696 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1c624-e858-4a0e-9648-a6333235ca61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.782380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14541c5-bb51-458d-8a61-77aec548d33e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:27.785579 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1c624-e858-4a0e-9648-a6333235ca61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f1d1c624-e858-4a0e-9648-a6333235ca61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:27.790845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:27.791769 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:27.792687 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 270/538] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:27.793477 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a79c3-52ab-4b87-9005-0ce90a1f1b21 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.794416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a79c3-52ab-4b87-9005-0ce90a1f1b21 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.794416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a79c3-52ab-4b87-9005-0ce90a1f1b21 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.799096 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 271/539] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:27.799096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa46d9-c656-4058-9760-2a19bd7ce972 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.799096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa46d9-c656-4058-9760-2a19bd7ce972 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.799096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa46d9-c656-4058-9760-2a19bd7ce972 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.799422 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 272/540] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:27.800296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa46d9-c656-4058-9760-2a19bd7ce972 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:27.829382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d88ea8-c50f-4e21-ba18-e0ccdc153780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:27.830090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d88ea8-c50f-4e21-ba18-e0ccdc153780 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.830436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d88ea8-c50f-4e21-ba18-e0ccdc153780 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.830885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d88ea8-c50f-4e21-ba18-e0ccdc153780 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.831563 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d88ea8-c50f-4e21-ba18-e0ccdc153780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:27.832192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d88ea8-c50f-4e21-ba18-e0ccdc153780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:27.833600 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d88ea8-c50f-4e21-ba18-e0ccdc153780 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:27.846113 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 269/541] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:27.846113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a40b0-75eb-4659-957b-7fb4f78b2e66 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.846113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a40b0-75eb-4659-957b-7fb4f78b2e66 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.846113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a40b0-75eb-4659-957b-7fb4f78b2e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.861745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cd0a40b0-75eb-4659-957b-7fb4f78b2e66 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:27.862976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cd0a40b0-75eb-4659-957b-7fb4f78b2e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:27.905479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa46d9-c656-4058-9760-2a19bd7ce972 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:27.920014 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:27.933973 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:27.935112 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 270/542] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:27.936641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e42d4f-aff1-433f-ac87-5ec8fa50af90 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.937055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e42d4f-aff1-433f-ac87-5ec8fa50af90 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.937455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e42d4f-aff1-433f-ac87-5ec8fa50af90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.942565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f682e-ad1e-4221-b569-920ad16c48cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:27.943264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f682e-ad1e-4221-b569-920ad16c48cf None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:27.943735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f682e-ad1e-4221-b569-920ad16c48cf None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:27.943928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f682e-ad1e-4221-b569-920ad16c48cf None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:27.944773 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f682e-ad1e-4221-b569-920ad16c48cf None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:27.945077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f682e-ad1e-4221-b569-920ad16c48cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:27.947473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f682e-ad1e-4221-b569-920ad16c48cf None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:27.957221 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 273/543] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:27.957675 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-10e42d4f-aff1-433f-ac87-5ec8fa50af90 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:27.958060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f682e-ad1e-4221-b569-920ad16c48cf None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:27.959418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-10e42d4f-aff1-433f-ac87-5ec8fa50af90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:28.029586 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 271/544] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 08:35:28.030812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e42d4f-aff1-433f-ac87-5ec8fa50af90 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.104416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f682e-ad1e-4221-b569-920ad16c48cf None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.136439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e42d4f-aff1-433f-ac87-5ec8fa50af90 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.154531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:28.172671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36323c9-b080-4aac-869a-868bde6410dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.173530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36323c9-b080-4aac-869a-868bde6410dd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.173953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36323c9-b080-4aac-869a-868bde6410dd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.174252 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc61514-f00d-445c-a335-9f27b2413be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.174538 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36323c9-b080-4aac-869a-868bde6410dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.174915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc61514-f00d-445c-a335-9f27b2413be8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.175191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc61514-f00d-445c-a335-9f27b2413be8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.175490 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc61514-f00d-445c-a335-9f27b2413be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.185511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36323c9-b080-4aac-869a-868bde6410dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:28.187148 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36323c9-b080-4aac-869a-868bde6410dd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:28.195199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc61514-f00d-445c-a335-9f27b2413be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:28.196647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc61514-f00d-445c-a335-9f27b2413be8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f36323c9-b080-4aac-869a-868bde6410dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:28.202853 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:28.203984 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:28.205101 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:28.205101 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:28.205101 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:28.206839 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 272/545] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:28.212335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36323c9-b080-4aac-869a-868bde6410dd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7fc61514-f00d-445c-a335-9f27b2413be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:28.231267 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:28.232419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:28.233534 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:28.233534 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:28.233534 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:28.235818 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 274/546] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:28.236977 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc61514-f00d-445c-a335-9f27b2413be8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.297697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:28.360179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36323c9-b080-4aac-869a-868bde6410dd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.370754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc61514-f00d-445c-a335-9f27b2413be8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.398958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fda747-f2f3-4eda-b68e-d2a3beca0589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.400275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fda747-f2f3-4eda-b68e-d2a3beca0589 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.400964 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fda747-f2f3-4eda-b68e-d2a3beca0589 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.401408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fda747-f2f3-4eda-b68e-d2a3beca0589 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.402339 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fda747-f2f3-4eda-b68e-d2a3beca0589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:28.402850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fda747-f2f3-4eda-b68e-d2a3beca0589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:28.404408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fda747-f2f3-4eda-b68e-d2a3beca0589 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:28.404625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f2134-a73a-4df7-bf48-7db883a340be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.405264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f2134-a73a-4df7-bf48-7db883a340be None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.405541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f2134-a73a-4df7-bf48-7db883a340be None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.405811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f2134-a73a-4df7-bf48-7db883a340be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.416209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f2134-a73a-4df7-bf48-7db883a340be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:28.419362 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 273/547] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:28.420451 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fda747-f2f3-4eda-b68e-d2a3beca0589 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2c6f2134-a73a-4df7-bf48-7db883a340be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:28.421542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:28.422476 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:28.423861 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:28.427273 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 275/548] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:35:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:28.428079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f2134-a73a-4df7-bf48-7db883a340be None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.454319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:28.528051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fda747-f2f3-4eda-b68e-d2a3beca0589 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.543088 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f2134-a73a-4df7-bf48-7db883a340be None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.544995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:28.564030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81e619-2df8-4301-ae60-c11dc0cc2c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.564846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81e619-2df8-4301-ae60-c11dc0cc2c74 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.565292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81e619-2df8-4301-ae60-c11dc0cc2c74 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.565725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81e619-2df8-4301-ae60-c11dc0cc2c74 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.566519 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81e619-2df8-4301-ae60-c11dc0cc2c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:28.566853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81e619-2df8-4301-ae60-c11dc0cc2c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:28.568407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81e619-2df8-4301-ae60-c11dc0cc2c74 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:28.578611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f25680-3441-44a1-8397-7a4816bf551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.579379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f25680-3441-44a1-8397-7a4816bf551c None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.579656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f25680-3441-44a1-8397-7a4816bf551c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.579928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f25680-3441-44a1-8397-7a4816bf551c None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.580575 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f25680-3441-44a1-8397-7a4816bf551c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:28.581941 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f25680-3441-44a1-8397-7a4816bf551c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:28.586755 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 274/549] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:28.586755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81e619-2df8-4301-ae60-c11dc0cc2c74 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.599894 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 276/550] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:35:28 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:28.601459 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f25680-3441-44a1-8397-7a4816bf551c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.694237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:28.706570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81e619-2df8-4301-ae60-c11dc0cc2c74 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.753309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bc3a1-e1d4-492d-953d-87d331f94a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.753638 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f25680-3441-44a1-8397-7a4816bf551c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.754632 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bc3a1-e1d4-492d-953d-87d331f94a1c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.755291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bc3a1-e1d4-492d-953d-87d331f94a1c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.755772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bc3a1-e1d4-492d-953d-87d331f94a1c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.765706 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bc3a1-e1d4-492d-953d-87d331f94a1c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:28.767699 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bc3a1-e1d4-492d-953d-87d331f94a1c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-191bc3a1-e1d4-492d-953d-87d331f94a1c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:28.781335 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:28.782336 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 08:35:28.783328 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:28.788984 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 275/551] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Oct 24 08:35:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:28.789756 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bc3a1-e1d4-492d-953d-87d331f94a1c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.795011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23744aa-4fc8-4923-a404-d5d07e364033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.795648 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23744aa-4fc8-4923-a404-d5d07e364033 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.795921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23744aa-4fc8-4923-a404-d5d07e364033 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.796195 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23744aa-4fc8-4923-a404-d5d07e364033 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.796864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23744aa-4fc8-4923-a404-d5d07e364033 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:28.797166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23744aa-4fc8-4923-a404-d5d07e364033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:28.798548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23744aa-4fc8-4923-a404-d5d07e364033 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:28.809743 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 277/552] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:28.810401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23744aa-4fc8-4923-a404-d5d07e364033 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.846433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:28.907034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bc3a1-e1d4-492d-953d-87d331f94a1c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.949435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23744aa-4fc8-4923-a404-d5d07e364033 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:28.956582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97459be2-e426-4292-94ff-4fc52064c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.957316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97459be2-e426-4292-94ff-4fc52064c850 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.957671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97459be2-e426-4292-94ff-4fc52064c850 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.958513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97459be2-e426-4292-94ff-4fc52064c850 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.967230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97459be2-e426-4292-94ff-4fc52064c850 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:28.968880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97459be2-e426-4292-94ff-4fc52064c850 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-97459be2-e426-4292-94ff-4fc52064c850 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:28.980719 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:28.981732 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:35:28.982684 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:28.985312 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 276/553] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Oct 24 08:35:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:28.986378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97459be2-e426-4292-94ff-4fc52064c850 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:28.994755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4e13c-8e00-474f-a952-91df050530d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:28.995614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4e13c-8e00-474f-a952-91df050530d5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:28.995964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4e13c-8e00-474f-a952-91df050530d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:28.996303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4e13c-8e00-474f-a952-91df050530d5 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:28.997103 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4e13c-8e00-474f-a952-91df050530d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:28.997474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4e13c-8e00-474f-a952-91df050530d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:28.999008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4e13c-8e00-474f-a952-91df050530d5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.000379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:29.026805 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 278/554] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Oct 24 08:35:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:29.028040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4e13c-8e00-474f-a952-91df050530d5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:29.121315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97459be2-e426-4292-94ff-4fc52064c850 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:29.122171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:29.144453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4e13c-8e00-474f-a952-91df050530d5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:29.164995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f27599-7160-4924-97ca-9b442ae81d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:29.165644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f27599-7160-4924-97ca-9b442ae81d4a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.165913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f27599-7160-4924-97ca-9b442ae81d4a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.166186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f27599-7160-4924-97ca-9b442ae81d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.183322 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f27599-7160-4924-97ca-9b442ae81d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:29.185520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f27599-7160-4924-97ca-9b442ae81d4a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.187694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be7d4-e20a-4f7e-a65d-bed8e5310285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:29.188382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be7d4-e20a-4f7e-a65d-bed8e5310285 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.188727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be7d4-e20a-4f7e-a65d-bed8e5310285 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.189069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be7d4-e20a-4f7e-a65d-bed8e5310285 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.189853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be7d4-e20a-4f7e-a65d-bed8e5310285 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:29.190229 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be7d4-e20a-4f7e-a65d-bed8e5310285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:29.191787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be7d4-e20a-4f7e-a65d-bed8e5310285 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c8f27599-7160-4924-97ca-9b442ae81d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:29.204861 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:29.205845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:29.206801 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:29.206801 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:29.206801 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:29.210861 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 277/555] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:29.211874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f27599-7160-4924-97ca-9b442ae81d4a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:29.216072 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 279/556] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:29.216990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be7d4-e20a-4f7e-a65d-bed8e5310285 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:29.293070 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:29.335152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be7d4-e20a-4f7e-a65d-bed8e5310285 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:29.353537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f27599-7160-4924-97ca-9b442ae81d4a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:29.379064 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11707752-ee20-4cd1-bc84-cbc12b17035e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:29.379708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707752-ee20-4cd1-bc84-cbc12b17035e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.380035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707752-ee20-4cd1-bc84-cbc12b17035e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.380308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707752-ee20-4cd1-bc84-cbc12b17035e None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.380954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11707752-ee20-4cd1-bc84-cbc12b17035e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:29.381302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11707752-ee20-4cd1-bc84-cbc12b17035e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:29.382762 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11707752-ee20-4cd1-bc84-cbc12b17035e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.385618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:29.396451 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 280/557] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:29.396802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11707752-ee20-4cd1-bc84-cbc12b17035e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:29.404028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9e04b-7983-4f2e-ad75-dc74eda9cab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:29.404929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e04b-7983-4f2e-ad75-dc74eda9cab3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.405387 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e04b-7983-4f2e-ad75-dc74eda9cab3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.406355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e04b-7983-4f2e-ad75-dc74eda9cab3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.421535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9e04b-7983-4f2e-ad75-dc74eda9cab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:29.423723 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9e04b-7983-4f2e-ad75-dc74eda9cab3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-17d9e04b-7983-4f2e-ad75-dc74eda9cab3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:29.438235 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:29.439237 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:29.440157 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:29.440157 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:29.440157 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:29.440157 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 278/558] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:29.441181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d9701-dbe9-4f35-ab5e-93f1aa0ec9f0 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.441181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d9701-dbe9-4f35-ab5e-93f1aa0ec9f0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.441181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d9701-dbe9-4f35-ab5e-93f1aa0ec9f0 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.446886 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 279/559] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:29.446886 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d9701-dbe9-4f35-ab5e-93f1aa0ec9f0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:29.503766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11707752-ee20-4cd1-bc84-cbc12b17035e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:29.536944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e2604-415a-4b6d-b81f-ab5412e4ed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:29.537561 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e2604-415a-4b6d-b81f-ab5412e4ed63 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.537837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e2604-415a-4b6d-b81f-ab5412e4ed63 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.538116 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e2604-415a-4b6d-b81f-ab5412e4ed63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.554395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d9701-dbe9-4f35-ab5e-93f1aa0ec9f0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:29.567313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:29.596515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fc83d-b29a-4723-817b-0966289572f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:29.597161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fc83d-b29a-4723-817b-0966289572f0 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.597442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fc83d-b29a-4723-817b-0966289572f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.597709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fc83d-b29a-4723-817b-0966289572f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.601898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:29.607818 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fc83d-b29a-4723-817b-0966289572f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:29.609337 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fc83d-b29a-4723-817b-0966289572f0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.609621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e2604-415a-4b6d-b81f-ab5412e4ed63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:29.611219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e2604-415a-4b6d-b81f-ab5412e4ed63 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e23fc83d-b29a-4723-817b-0966289572f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:29.621049 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:29.621989 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 280/560] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:29.628128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fc83d-b29a-4723-817b-0966289572f0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:29.711855 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 281/561] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:35:29.713417 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e72a53-28cb-469e-9dad-633d3d0b75fd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.713756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e72a53-28cb-469e-9dad-633d3d0b75fd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.714155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e72a53-28cb-469e-9dad-633d3d0b75fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.731952 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-54e72a53-28cb-469e-9dad-633d3d0b75fd None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:29.733565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-54e72a53-28cb-469e-9dad-633d3d0b75fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:29.735720 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fc83d-b29a-4723-817b-0966289572f0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:29.770848 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d1f62-3374-40f7-ae80-7a1eb0e4d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:29.771375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d1f62-3374-40f7-ae80-7a1eb0e4d864 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.771652 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d1f62-3374-40f7-ae80-7a1eb0e4d864 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.771921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d1f62-3374-40f7-ae80-7a1eb0e4d864 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.772557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d1f62-3374-40f7-ae80-7a1eb0e4d864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:29.772828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d1f62-3374-40f7-ae80-7a1eb0e4d864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:29.774251 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d1f62-3374-40f7-ae80-7a1eb0e4d864 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.791710 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 281/562] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:29.791710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d1f62-3374-40f7-ae80-7a1eb0e4d864 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:29.804535 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 282/563] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:29.805371 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e72a53-28cb-469e-9dad-633d3d0b75fd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:29.831382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:29.832513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:29.910624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e72a53-28cb-469e-9dad-633d3d0b75fd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:29.915115 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d1f62-3374-40f7-ae80-7a1eb0e4d864 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:29.945918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e37f0-3441-4e3d-a880-9ce133e90d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:29.947130 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e37f0-3441-4e3d-a880-9ce133e90d9c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.947721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e37f0-3441-4e3d-a880-9ce133e90d9c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.948384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e37f0-3441-4e3d-a880-9ce133e90d9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.950093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cc39a-8693-4b6f-8209-08f358b58cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:29.950737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cc39a-8693-4b6f-8209-08f358b58cda None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:29.951047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cc39a-8693-4b6f-8209-08f358b58cda None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:29.951321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cc39a-8693-4b6f-8209-08f358b58cda None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:29.952042 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cc39a-8693-4b6f-8209-08f358b58cda None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:29.952333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cc39a-8693-4b6f-8209-08f358b58cda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:29.953757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cc39a-8693-4b6f-8209-08f358b58cda None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.960751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e37f0-3441-4e3d-a880-9ce133e90d9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:29.962443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e37f0-3441-4e3d-a880-9ce133e90d9c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:29.976554 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 282/564] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:29.977840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cc39a-8693-4b6f-8209-08f358b58cda None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7c8e37f0-3441-4e3d-a880-9ce133e90d9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:29.978411 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:29.979377 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:29.980391 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:29.980391 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:29.980391 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:29.986693 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 283/565] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:29.987800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e37f0-3441-4e3d-a880-9ce133e90d9c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.068112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:30.068686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:30.082702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cc39a-8693-4b6f-8209-08f358b58cda None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.095830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e37f0-3441-4e3d-a880-9ce133e90d9c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.115521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac820d87-6229-4d6a-8b79-f9651f51fabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.116139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac820d87-6229-4d6a-8b79-f9651f51fabf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.116412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac820d87-6229-4d6a-8b79-f9651f51fabf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.116679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac820d87-6229-4d6a-8b79-f9651f51fabf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.123347 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac820d87-6229-4d6a-8b79-f9651f51fabf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.124748 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac820d87-6229-4d6a-8b79-f9651f51fabf None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.133842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6214207-dc07-44d1-b1fa-abe5589605da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.134468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6214207-dc07-44d1-b1fa-abe5589605da None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.134752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6214207-dc07-44d1-b1fa-abe5589605da None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.135035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6214207-dc07-44d1-b1fa-abe5589605da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-ac820d87-6229-4d6a-8b79-f9651f51fabf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:30.135524 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:30.136467 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 283/566] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:30.137388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac820d87-6229-4d6a-8b79-f9651f51fabf None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.150703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6214207-dc07-44d1-b1fa-abe5589605da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.152209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6214207-dc07-44d1-b1fa-abe5589605da None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e6214207-dc07-44d1-b1fa-abe5589605da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:30.166302 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:30.167335 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:30.168337 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:30.168337 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:30.168337 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:30.168337 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 284/567] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:30.168947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6214207-dc07-44d1-b1fa-abe5589605da None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.244342 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac820d87-6229-4d6a-8b79-f9651f51fabf None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.274493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6214207-dc07-44d1-b1fa-abe5589605da None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.289521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d63421-42ce-42c0-bc5e-76c3c9532853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.290130 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d63421-42ce-42c0-bc5e-76c3c9532853 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.290401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d63421-42ce-42c0-bc5e-76c3c9532853 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.290667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d63421-42ce-42c0-bc5e-76c3c9532853 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.291350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d63421-42ce-42c0-bc5e-76c3c9532853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:30.291620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d63421-42ce-42c0-bc5e-76c3c9532853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.293031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d63421-42ce-42c0-bc5e-76c3c9532853 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.305116 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 284/568] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:30.305911 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d63421-42ce-42c0-bc5e-76c3c9532853 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.308247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:30.316367 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a026372-aaa9-4f13-91dd-d41782ac055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.317002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a026372-aaa9-4f13-91dd-d41782ac055e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.317275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a026372-aaa9-4f13-91dd-d41782ac055e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.317539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a026372-aaa9-4f13-91dd-d41782ac055e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.318099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a026372-aaa9-4f13-91dd-d41782ac055e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.318803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:30.319496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a026372-aaa9-4f13-91dd-d41782ac055e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-8a026372-aaa9-4f13-91dd-d41782ac055e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:30.331954 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:30.333068 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:30.335707 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 285/569] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:30.336528 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a026372-aaa9-4f13-91dd-d41782ac055e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.409057 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d63421-42ce-42c0-bc5e-76c3c9532853 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.435982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a026372-aaa9-4f13-91dd-d41782ac055e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.448314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ae40a-781b-4be6-864e-98745e28b98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.448914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ae40a-781b-4be6-864e-98745e28b98e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.449185 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ae40a-781b-4be6-864e-98745e28b98e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.449446 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ae40a-781b-4be6-864e-98745e28b98e None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.450121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ae40a-781b-4be6-864e-98745e28b98e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:30.450419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ae40a-781b-4be6-864e-98745e28b98e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.451888 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ae40a-781b-4be6-864e-98745e28b98e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.465143 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 285/570] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:30.466068 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ae40a-781b-4be6-864e-98745e28b98e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.477559 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294e591-1adf-4c03-978f-7120157e9633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.478520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294e591-1adf-4c03-978f-7120157e9633 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.478826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294e591-1adf-4c03-978f-7120157e9633 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.479092 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294e591-1adf-4c03-978f-7120157e9633 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.480379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294e591-1adf-4c03-978f-7120157e9633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:30.480686 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294e591-1adf-4c03-978f-7120157e9633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.482117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294e591-1adf-4c03-978f-7120157e9633 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.497828 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 286/571] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:30.497828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294e591-1adf-4c03-978f-7120157e9633 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.541456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:30.549943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:30.579149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ae40a-781b-4be6-864e-98745e28b98e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.592474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294e591-1adf-4c03-978f-7120157e9633 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.616964 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20350098-e511-45b0-a405-31c1c6151be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.617648 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20350098-e511-45b0-a405-31c1c6151be9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.617929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20350098-e511-45b0-a405-31c1c6151be9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.618200 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20350098-e511-45b0-a405-31c1c6151be9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.625269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20350098-e511-45b0-a405-31c1c6151be9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.626730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20350098-e511-45b0-a405-31c1c6151be9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.627029 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce8ab2-faa2-4c2e-adec-9969a750d9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.627635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce8ab2-faa2-4c2e-adec-9969a750d9fa None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.627903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce8ab2-faa2-4c2e-adec-9969a750d9fa None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.628170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce8ab2-faa2-4c2e-adec-9969a750d9fa None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.628951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce8ab2-faa2-4c2e-adec-9969a750d9fa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:30.629307 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce8ab2-faa2-4c2e-adec-9969a750d9fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.630891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce8ab2-faa2-4c2e-adec-9969a750d9fa None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.640294 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 287/572] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:30.640294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce8ab2-faa2-4c2e-adec-9969a750d9fa None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-20350098-e511-45b0-a405-31c1c6151be9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:30.640531 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:30.641359 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 286/573] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:30.642288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20350098-e511-45b0-a405-31c1c6151be9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.750245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce8ab2-faa2-4c2e-adec-9969a750d9fa None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.752507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20350098-e511-45b0-a405-31c1c6151be9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.770293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:30.782305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:30.789125 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeb18e-ee02-41ab-8226-dee78bc389c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.789782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeb18e-ee02-41ab-8226-dee78bc389c1 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.790054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeb18e-ee02-41ab-8226-dee78bc389c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.790319 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeb18e-ee02-41ab-8226-dee78bc389c1 None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/771fc0da7a8749f0b80ce0630951500a/roles/cb7d27022ded4ea2afd3d5ebaefbf429` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.791653 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979abd5-08ad-426d-882a-40c2a207beb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.792334 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979abd5-08ad-426d-882a-40c2a207beb0 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.792615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979abd5-08ad-426d-882a-40c2a207beb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.792906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979abd5-08ad-426d-882a-40c2a207beb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.799655 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979abd5-08ad-426d-882a-40c2a207beb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.801095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979abd5-08ad-426d-882a-40c2a207beb0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.805482 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeb18e-ee02-41ab-8226-dee78bc389c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=771fc0da7a8749f0b80ce0630951500a, role_id=cb7d27022ded4ea2afd3d5ebaefbf429)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.806945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeb18e-ee02-41ab-8226-dee78bc389c1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-6979abd5-08ad-426d-882a-40c2a207beb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:30.812483 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:30.813588 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:30.814636 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:30.814636 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:30.814636 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:30.814636 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:30.814636 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:30.814636 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:30.814636 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:30.814636 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:30.814636 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 287/574] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:30.815180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979abd5-08ad-426d-882a-40c2a207beb0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.827298 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 288/575] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:35:30 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/771fc0da7a8749f0b80ce0630951500a/roles/cb7d27022ded4ea2afd3d5ebaefbf429 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:35:30.828885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeb18e-ee02-41ab-8226-dee78bc389c1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:30.941266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeb18e-ee02-41ab-8226-dee78bc389c1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.941971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979abd5-08ad-426d-882a-40c2a207beb0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:30.977801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608031a-7ae1-48f6-a6d4-91ddefb576d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.978418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608031a-7ae1-48f6-a6d4-91ddefb576d2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.978699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608031a-7ae1-48f6-a6d4-91ddefb576d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.978992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608031a-7ae1-48f6-a6d4-91ddefb576d2 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.979774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608031a-7ae1-48f6-a6d4-91ddefb576d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:30.980432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608031a-7ae1-48f6-a6d4-91ddefb576d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.981933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608031a-7ae1-48f6-a6d4-91ddefb576d2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.984865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62629ee-a85b-4882-af97-557a6ae52c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:30.985487 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62629ee-a85b-4882-af97-557a6ae52c56 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.985764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62629ee-a85b-4882-af97-557a6ae52c56 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.986036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62629ee-a85b-4882-af97-557a6ae52c56 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:30.986680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62629ee-a85b-4882-af97-557a6ae52c56 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:30.987015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62629ee-a85b-4882-af97-557a6ae52c56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:30.988407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62629ee-a85b-4882-af97-557a6ae52c56 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:30.999340 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 289/576] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:30.999539 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 288/577] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:30.999539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2df48-b4c0-420b-9aed-e774c7e866a4 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:30.999539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2df48-b4c0-420b-9aed-e774c7e866a4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:30.999539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2df48-b4c0-420b-9aed-e774c7e866a4 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:31.000440 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608031a-7ae1-48f6-a6d4-91ddefb576d2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:31.002948 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 289/578] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:31.004529 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2df48-b4c0-420b-9aed-e774c7e866a4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:31.032627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:31.067678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:31.110039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608031a-7ae1-48f6-a6d4-91ddefb576d2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:31.121020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2df48-b4c0-420b-9aed-e774c7e866a4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:31.144206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788dd28-3f69-48e3-9591-f3a9b2e725b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:31.144830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788dd28-3f69-48e3-9591-f3a9b2e725b0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:31.145113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788dd28-3f69-48e3-9591-f3a9b2e725b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:31.145392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788dd28-3f69-48e3-9591-f3a9b2e725b0 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:31.146069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788dd28-3f69-48e3-9591-f3a9b2e725b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:31.146378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788dd28-3f69-48e3-9591-f3a9b2e725b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:31.147862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788dd28-3f69-48e3-9591-f3a9b2e725b0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:31.156003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb340f-e1a3-46cd-991a-ca46da853b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:31.156630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb340f-e1a3-46cd-991a-ca46da853b2a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:31.156904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb340f-e1a3-46cd-991a-ca46da853b2a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:31.157175 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb340f-e1a3-46cd-991a-ca46da853b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:31.157897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb340f-e1a3-46cd-991a-ca46da853b2a None admin] RBAC: Adding query filter params (role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=f257d9b665714bff8b4ceb4fc5dc8387) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:31.158191 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb340f-e1a3-46cd-991a-ca46da853b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:31.159650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb340f-e1a3-46cd-991a-ca46da853b2a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:31.166906 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 290/579] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:31.166906 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788dd28-3f69-48e3-9591-f3a9b2e725b0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:31.171174 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 290/580] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:35:31 2022] GET /identity/v3/role_assignments?user.id=f257d9b665714bff8b4ceb4fc5dc8387&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:31.175521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f203f6-2c62-4ac3-987e-a298cc2f857c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:31.175521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f203f6-2c62-4ac3-987e-a298cc2f857c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:31.175521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f203f6-2c62-4ac3-987e-a298cc2f857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:31.192251 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-50f203f6-2c62-4ac3-987e-a298cc2f857c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:31.193456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-50f203f6-2c62-4ac3-987e-a298cc2f857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:31.264866 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:31.272320 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 291/581] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:31.273518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f203f6-2c62-4ac3-987e-a298cc2f857c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:31.291216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788dd28-3f69-48e3-9591-f3a9b2e725b0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:31.352566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75d73c-57fd-4547-9875-8cdd90fb4589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:31.353382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75d73c-57fd-4547-9875-8cdd90fb4589 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:31.353757 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75d73c-57fd-4547-9875-8cdd90fb4589 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:31.354133 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75d73c-57fd-4547-9875-8cdd90fb4589 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:31.355046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75d73c-57fd-4547-9875-8cdd90fb4589 None admin] RBAC: Adding query filter params (interface=public, service_id=e9e1011cc07e4de09debebf760cf2d7b, region_id=RegionOne) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:31.355156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:31.355957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75d73c-57fd-4547-9875-8cdd90fb4589 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:31.357726 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75d73c-57fd-4547-9875-8cdd90fb4589 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:31.375187 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 291/582] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Oct 24 08:35:31 2022] GET /identity/v3/endpoints?service_id=e9e1011cc07e4de09debebf760cf2d7b&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:31.382705 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75d73c-57fd-4547-9875-8cdd90fb4589 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:31.453742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f203f6-2c62-4ac3-987e-a298cc2f857c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:31.498801 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11a66d-9957-4b53-8507-3ee3b1db9854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:31.499651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11a66d-9957-4b53-8507-3ee3b1db9854 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:31.500202 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11a66d-9957-4b53-8507-3ee3b1db9854 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:31.500584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11a66d-9957-4b53-8507-3ee3b1db9854 None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/dbf288ceecac4fec8b2161c55a61a8ba/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:31.503374 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11a66d-9957-4b53-8507-3ee3b1db9854 None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=dbf288ceecac4fec8b2161c55a61a8ba, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:31.505200 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11a66d-9957-4b53-8507-3ee3b1db9854 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:31.529124 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 292/583] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:35:31 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/dbf288ceecac4fec8b2161c55a61a8ba/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:35:31.530219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75d73c-57fd-4547-9875-8cdd90fb4589 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:31.563845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11a66d-9957-4b53-8507-3ee3b1db9854 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:31.593877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e639c5a-5005-41f0-9a5b-bdac7240b180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:31.594522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e639c5a-5005-41f0-9a5b-bdac7240b180 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:31.594833 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e639c5a-5005-41f0-9a5b-bdac7240b180 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:31.595726 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e639c5a-5005-41f0-9a5b-bdac7240b180 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:31.596444 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e639c5a-5005-41f0-9a5b-bdac7240b180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:31.598105 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e639c5a-5005-41f0-9a5b-bdac7240b180 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:31.606411 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 292/584] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Oct 24 08:35:31 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:31.698384 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11a66d-9957-4b53-8507-3ee3b1db9854 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:31.707670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:31.741340 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a4cf-506c-4ebc-8470-43dae28cee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:31.742334 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a4cf-506c-4ebc-8470-43dae28cee83 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:31.742802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a4cf-506c-4ebc-8470-43dae28cee83 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:31.743311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a4cf-506c-4ebc-8470-43dae28cee83 None admin] PATH_INFO: `/v3/services/network` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:31.744068 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5a4cf-506c-4ebc-8470-43dae28cee83 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:31.745686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5a4cf-506c-4ebc-8470-43dae28cee83 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-90a5a4cf-506c-4ebc-8470-43dae28cee83 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:31.756149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:31.757302 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 08:35:31.758212 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:31.761390 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 293/585] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:31 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:31.767689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e639c5a-5005-41f0-9a5b-bdac7240b180 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:31.798624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:31.899987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e639c5a-5005-41f0-9a5b-bdac7240b180 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:31.940074 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3411e-aac3-462b-a29f-8978830bfd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:31.940702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3411e-aac3-462b-a29f-8978830bfd4a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:31.940981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3411e-aac3-462b-a29f-8978830bfd4a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:31.941256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3411e-aac3-462b-a29f-8978830bfd4a None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:31.941889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3411e-aac3-462b-a29f-8978830bfd4a None admin] RBAC: Adding query filter params (name=network) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:31.942194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3411e-aac3-462b-a29f-8978830bfd4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:31.943608 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3411e-aac3-462b-a29f-8978830bfd4a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:31.949064 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 293/586] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:31.955450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a4cf-506c-4ebc-8470-43dae28cee83 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:32.112658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a4cf-506c-4ebc-8470-43dae28cee83 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:32.152441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8769e3-450c-49d0-a58d-18a53fb4e74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:32.153288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8769e3-450c-49d0-a58d-18a53fb4e74e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:32.153710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8769e3-450c-49d0-a58d-18a53fb4e74e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:32.154116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8769e3-450c-49d0-a58d-18a53fb4e74e None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:32.154896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8769e3-450c-49d0-a58d-18a53fb4e74e None admin] RBAC: Adding query filter params (type=network) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:32.155368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8769e3-450c-49d0-a58d-18a53fb4e74e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:32.156926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8769e3-450c-49d0-a58d-18a53fb4e74e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:32.166766 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 294/587] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:32.173083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3411e-aac3-462b-a29f-8978830bfd4a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:32.295025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:32.313428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3411e-aac3-462b-a29f-8978830bfd4a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:32.356873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ca300-c03a-414e-b329-b095db082d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:32.357507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca300-c03a-414e-b329-b095db082d70 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:32.357790 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca300-c03a-414e-b329-b095db082d70 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:32.358073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca300-c03a-414e-b329-b095db082d70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:32.358792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ca300-c03a-414e-b329-b095db082d70 None admin] RBAC: Adding query filter params (interface=public, service_id=0c4a978536204f83b24078419aa9a0fb, region_id=RegionOne) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:32.359106 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ca300-c03a-414e-b329-b095db082d70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:32.360484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ca300-c03a-414e-b329-b095db082d70 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:32.370943 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 294/588] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Oct 24 08:35:32 2022] GET /identity/v3/endpoints?service_id=0c4a978536204f83b24078419aa9a0fb&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:32.376434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8769e3-450c-49d0-a58d-18a53fb4e74e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:32.428993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:32.507770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8769e3-450c-49d0-a58d-18a53fb4e74e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:32.541856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c0d38-d36b-477e-994c-1d89de54d6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:32.542587 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c0d38-d36b-477e-994c-1d89de54d6dd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:32.542894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c0d38-d36b-477e-994c-1d89de54d6dd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:32.543166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c0d38-d36b-477e-994c-1d89de54d6dd None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:32.543862 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c0d38-d36b-477e-994c-1d89de54d6dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:32.545237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c0d38-d36b-477e-994c-1d89de54d6dd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:32.550819 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 295/589] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Oct 24 08:35:32 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:33.276674 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ae666-0245-44c2-a8fd-8228252a788c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:33.276994 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ae666-0245-44c2-a8fd-8228252a788c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:33.277269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ae666-0245-44c2-a8fd-8228252a788c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:33.278876 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 295/590] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:33.284775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d445d4c-e2a8-46bf-84ce-e5ca3f1becbb None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:33.285190 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d445d4c-e2a8-46bf-84ce-e5ca3f1becbb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:33.285360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d445d4c-e2a8-46bf-84ce-e5ca3f1becbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:33.310080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2d445d4c-e2a8-46bf-84ce-e5ca3f1becbb None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:33.311281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2d445d4c-e2a8-46bf-84ce-e5ca3f1becbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:33.386072 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 296/591] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:33.711162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ae666-0245-44c2-a8fd-8228252a788c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:33.842214 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ae666-0245-44c2-a8fd-8228252a788c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:33.883719 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1b60a-2e56-4790-9cad-ac68124f273b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:33.884356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1b60a-2e56-4790-9cad-ac68124f273b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:33.884630 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1b60a-2e56-4790-9cad-ac68124f273b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:33.884898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1b60a-2e56-4790-9cad-ac68124f273b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:33.895447 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1b60a-2e56-4790-9cad-ac68124f273b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-d2d1b60a-2e56-4790-9cad-ac68124f273b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:33.900300 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:33.901288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:33.902378 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 296/592] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:35:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:33.909680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d445d4c-e2a8-46bf-84ce-e5ca3f1becbb None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:33.940900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:34.032170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d445d4c-e2a8-46bf-84ce-e5ca3f1becbb None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:34.068927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c3357-1b06-4ad5-b7ec-6747d9e691cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:34.069535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c3357-1b06-4ad5-b7ec-6747d9e691cf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.069816 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c3357-1b06-4ad5-b7ec-6747d9e691cf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.070072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c3357-1b06-4ad5-b7ec-6747d9e691cf None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.070743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c3357-1b06-4ad5-b7ec-6747d9e691cf None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:34.071051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c3357-1b06-4ad5-b7ec-6747d9e691cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:34.072409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c3357-1b06-4ad5-b7ec-6747d9e691cf None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:34.086163 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 297/593] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:34.092401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1b60a-2e56-4790-9cad-ac68124f273b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:34.096468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:34.188019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb028d25-dca5-49ff-aa62-e4bcea6717aa None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.188464 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb028d25-dca5-49ff-aa62-e4bcea6717aa None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.188828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb028d25-dca5-49ff-aa62-e4bcea6717aa None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.190670 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 298/594] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:35:34.196739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7635-b3b8-4aca-b0d2-b338f1a299c7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.197422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7635-b3b8-4aca-b0d2-b338f1a299c7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.197713 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7635-b3b8-4aca-b0d2-b338f1a299c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.217539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-039c7635-b3b8-4aca-b0d2-b338f1a299c7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:34.218727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-039c7635-b3b8-4aca-b0d2-b338f1a299c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:34.237359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1b60a-2e56-4790-9cad-ac68124f273b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:34.294110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a29d7e-dd5d-484b-bab3-9de40dd910a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:34.294768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a29d7e-dd5d-484b-bab3-9de40dd910a4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.295045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a29d7e-dd5d-484b-bab3-9de40dd910a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.295377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a29d7e-dd5d-484b-bab3-9de40dd910a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.306299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a29d7e-dd5d-484b-bab3-9de40dd910a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:34.307913 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a29d7e-dd5d-484b-bab3-9de40dd910a4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:34.312980 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 299/595] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:34.314813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9230f71-c523-4c22-8c87-7d4a2acb9808 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.315327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9230f71-c523-4c22-8c87-7d4a2acb9808 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.315807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9230f71-c523-4c22-8c87-7d4a2acb9808 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.317482 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 300/596] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:34.323244 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ebd51-4273-4207-b51b-75121ce0322b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.323792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ebd51-4273-4207-b51b-75121ce0322b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-08a29d7e-dd5d-484b-bab3-9de40dd910a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:34.323910 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:34.324810 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:34.325728 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:34.325728 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:34.325728 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:34.325728 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 297/597] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:34.326796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ebd51-4273-4207-b51b-75121ce0322b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.330498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a29d7e-dd5d-484b-bab3-9de40dd910a4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:34.345471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-868ebd51-4273-4207-b51b-75121ce0322b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:34.346702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-868ebd51-4273-4207-b51b-75121ce0322b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:34.361647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:34.390096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:34.417393 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 301/598] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:34.452432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a29d7e-dd5d-484b-bab3-9de40dd910a4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:34.490990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ad8da-4418-45ab-8444-12af253a62e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:34.491976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ad8da-4418-45ab-8444-12af253a62e2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.492461 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ad8da-4418-45ab-8444-12af253a62e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.492951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ad8da-4418-45ab-8444-12af253a62e2 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.493877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ad8da-4418-45ab-8444-12af253a62e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:34.494425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ad8da-4418-45ab-8444-12af253a62e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:34.496048 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ad8da-4418-45ab-8444-12af253a62e2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:34.508957 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 298/599] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:34.514654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ebd51-4273-4207-b51b-75121ce0322b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:34.615768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ad8da-4418-45ab-8444-12af253a62e2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:34.652102 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ebd51-4273-4207-b51b-75121ce0322b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:34.687831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d401c-044c-46dc-976d-da3c3e5ce812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:34.688503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d401c-044c-46dc-976d-da3c3e5ce812 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.688783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d401c-044c-46dc-976d-da3c3e5ce812 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.689061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d401c-044c-46dc-976d-da3c3e5ce812 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.702789 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d401c-044c-46dc-976d-da3c3e5ce812 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:34.702789 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d401c-044c-46dc-976d-da3c3e5ce812 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-235d401c-044c-46dc-976d-da3c3e5ce812 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:34.712511 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:34.713626 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:34.714683 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 302/600] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Oct 24 08:35:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:34.724734 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d401c-044c-46dc-976d-da3c3e5ce812 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:34.743079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:34.764412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ad8da-4418-45ab-8444-12af253a62e2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:34.782332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:34.802831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03346cb0-c07f-4bd2-8b3b-5eded6f8a229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:34.803838 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03346cb0-c07f-4bd2-8b3b-5eded6f8a229 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.804323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03346cb0-c07f-4bd2-8b3b-5eded6f8a229 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.804790 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03346cb0-c07f-4bd2-8b3b-5eded6f8a229 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.805552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03346cb0-c07f-4bd2-8b3b-5eded6f8a229 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:34.807140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03346cb0-c07f-4bd2-8b3b-5eded6f8a229 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-03346cb0-c07f-4bd2-8b3b-5eded6f8a229 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:34.816504 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:34.817521 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 08:35:34.818448 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:34.821247 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 299/601] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Oct 24 08:35:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:35:34.828780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03346cb0-c07f-4bd2-8b3b-5eded6f8a229 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:34.841311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d401c-044c-46dc-976d-da3c3e5ce812 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:34.877347 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953c0aa-63f3-464f-a1d0-2385d2d7d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:34.877870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c0aa-63f3-464f-a1d0-2385d2d7d825 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.878152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c0aa-63f3-464f-a1d0-2385d2d7d825 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.878424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c0aa-63f3-464f-a1d0-2385d2d7d825 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.879184 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953c0aa-63f3-464f-a1d0-2385d2d7d825 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:34.879493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953c0aa-63f3-464f-a1d0-2385d2d7d825 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:34.880928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953c0aa-63f3-464f-a1d0-2385d2d7d825 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:34.902755 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 303/602] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Oct 24 08:35:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:34.902755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd31b2-b44a-4f95-874f-1e905d0cbdbf None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.902755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd31b2-b44a-4f95-874f-1e905d0cbdbf None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.902755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd31b2-b44a-4f95-874f-1e905d0cbdbf None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.902755 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 304/603] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:34.902755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd31b2-b44a-4f95-874f-1e905d0cbdbf None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:34.949933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03346cb0-c07f-4bd2-8b3b-5eded6f8a229 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:34.977209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:34.983128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05673c1-9d1d-424d-b39a-8ac4ad75cb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:34.983754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05673c1-9d1d-424d-b39a-8ac4ad75cb2c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:34.984036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05673c1-9d1d-424d-b39a-8ac4ad75cb2c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:34.984308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05673c1-9d1d-424d-b39a-8ac4ad75cb2c None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:34.984935 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05673c1-9d1d-424d-b39a-8ac4ad75cb2c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:34.985241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05673c1-9d1d-424d-b39a-8ac4ad75cb2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:34.986621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05673c1-9d1d-424d-b39a-8ac4ad75cb2c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:34.991278 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 300/604] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:34.996490 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05673c1-9d1d-424d-b39a-8ac4ad75cb2c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.020229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd31b2-b44a-4f95-874f-1e905d0cbdbf None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:35.045723 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:35.066327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55032b9-5f44-4c50-abde-e1043ac81bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:35.067289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55032b9-5f44-4c50-abde-e1043ac81bd4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.067568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55032b9-5f44-4c50-abde-e1043ac81bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.067835 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55032b9-5f44-4c50-abde-e1043ac81bd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.078444 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55032b9-5f44-4c50-abde-e1043ac81bd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f55032b9-5f44-4c50-abde-e1043ac81bd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:35.083231 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:35.084185 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:35.085025 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 305/605] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:35:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:35.085776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31100491-4293-4f67-b091-9b2ed95af41d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.085776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31100491-4293-4f67-b091-9b2ed95af41d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.091698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31100491-4293-4f67-b091-9b2ed95af41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.105607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-31100491-4293-4f67-b091-9b2ed95af41d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:35.106829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-31100491-4293-4f67-b091-9b2ed95af41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:35.108059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05673c1-9d1d-424d-b39a-8ac4ad75cb2c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:35.145182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c18c82-6aca-4fb5-bb2b-d20b04f7558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:35.145708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c18c82-6aca-4fb5-bb2b-d20b04f7558c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.145980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c18c82-6aca-4fb5-bb2b-d20b04f7558c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.146244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c18c82-6aca-4fb5-bb2b-d20b04f7558c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.176862 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 306/606] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:35.177530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31100491-4293-4f67-b091-9b2ed95af41d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.230090 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c18c82-6aca-4fb5-bb2b-d20b04f7558c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:35.231577 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c18c82-6aca-4fb5-bb2b-d20b04f7558c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:35.234949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:35.279669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:35.290261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31100491-4293-4f67-b091-9b2ed95af41d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:35.324628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df202c9f-e053-4cf0-a19e-3c5684eb16e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:35.325882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df202c9f-e053-4cf0-a19e-3c5684eb16e9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.326305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df202c9f-e053-4cf0-a19e-3c5684eb16e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.326787 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df202c9f-e053-4cf0-a19e-3c5684eb16e9 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.327712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df202c9f-e053-4cf0-a19e-3c5684eb16e9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:35.328438 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df202c9f-e053-4cf0-a19e-3c5684eb16e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:35.330233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df202c9f-e053-4cf0-a19e-3c5684eb16e9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:35.346746 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 307/607] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:35.346746 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df202c9f-e053-4cf0-a19e-3c5684eb16e9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.347061 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 301/608] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:35:35.347061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f42e-6d6a-4817-9d57-ac4e8d2034e7 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.347061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f42e-6d6a-4817-9d57-ac4e8d2034e7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.347061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f42e-6d6a-4817-9d57-ac4e8d2034e7 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.347061 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 302/609] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:35.349224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f42e-6d6a-4817-9d57-ac4e8d2034e7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.460182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df202c9f-e053-4cf0-a19e-3c5684eb16e9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:35.484621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:35.489981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f42e-6d6a-4817-9d57-ac4e8d2034e7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:35.499315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462f361-6027-4726-a1bd-a95d293123ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:35.500283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462f361-6027-4726-a1bd-a95d293123ca None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.500593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462f361-6027-4726-a1bd-a95d293123ca None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.500864 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462f361-6027-4726-a1bd-a95d293123ca None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.501539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462f361-6027-4726-a1bd-a95d293123ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:35.502171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462f361-6027-4726-a1bd-a95d293123ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:35.503900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462f361-6027-4726-a1bd-a95d293123ca None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:35.519472 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 308/610] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:35.519472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfaa8b-6d36-4918-8129-465dae41415c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.519472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfaa8b-6d36-4918-8129-465dae41415c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.519472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfaa8b-6d36-4918-8129-465dae41415c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.530308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af282c8b-1c1d-40bd-aa1f-5a6499ad14cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:35.530981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af282c8b-1c1d-40bd-aa1f-5a6499ad14cd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.531259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af282c8b-1c1d-40bd-aa1f-5a6499ad14cd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.531532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af282c8b-1c1d-40bd-aa1f-5a6499ad14cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.538392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e4cfaa8b-6d36-4918-8129-465dae41415c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:35.539622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e4cfaa8b-6d36-4918-8129-465dae41415c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:35.548434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af282c8b-1c1d-40bd-aa1f-5a6499ad14cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:35.548434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af282c8b-1c1d-40bd-aa1f-5a6499ad14cd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:35.548434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-af282c8b-1c1d-40bd-aa1f-5a6499ad14cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:35.562781 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:35.563767 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:35.564674 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:35.564674 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:35.564674 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:35.565853 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 303/611] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:35.567231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af282c8b-1c1d-40bd-aa1f-5a6499ad14cd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.615906 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 309/612] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:35.617044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfaa8b-6d36-4918-8129-465dae41415c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.676531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af282c8b-1c1d-40bd-aa1f-5a6499ad14cd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:35.715736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4308-754a-4d40-95e4-b4293ddd7952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:35.716982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4308-754a-4d40-95e4-b4293ddd7952 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.717474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4308-754a-4d40-95e4-b4293ddd7952 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.717910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4b4308-754a-4d40-95e4-b4293ddd7952 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.718636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b4308-754a-4d40-95e4-b4293ddd7952 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:35.721042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4b4308-754a-4d40-95e4-b4293ddd7952 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:35.732414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfaa8b-6d36-4918-8129-465dae41415c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ee4b4308-754a-4d40-95e4-b4293ddd7952 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:35.732859 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:35.734015 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 08:35:35.735089 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:35.738639 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 304/613] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Oct 24 08:35:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:35.739631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4308-754a-4d40-95e4-b4293ddd7952 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.751346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:35.775084 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687c24b-5af3-4beb-b5f4-23dd4e5023c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:35.775918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687c24b-5af3-4beb-b5f4-23dd4e5023c7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.776345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687c24b-5af3-4beb-b5f4-23dd4e5023c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.776955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687c24b-5af3-4beb-b5f4-23dd4e5023c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.789518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687c24b-5af3-4beb-b5f4-23dd4e5023c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:35.791123 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687c24b-5af3-4beb-b5f4-23dd4e5023c7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-6687c24b-5af3-4beb-b5f4-23dd4e5023c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:35.807107 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:35.808283 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:35.809173 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:35.809173 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:35.809173 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:35.809576 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 310/614] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:35.810439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687c24b-5af3-4beb-b5f4-23dd4e5023c7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.823453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:35.867840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4b4308-754a-4d40-95e4-b4293ddd7952 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:35.902698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb87c-af17-4637-9355-8c3304d7c89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:35.903678 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb87c-af17-4637-9355-8c3304d7c89c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.904132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb87c-af17-4637-9355-8c3304d7c89c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.904576 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fb87c-af17-4637-9355-8c3304d7c89c None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.905388 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fb87c-af17-4637-9355-8c3304d7c89c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:35.905861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fb87c-af17-4637-9355-8c3304d7c89c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:35.907546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fb87c-af17-4637-9355-8c3304d7c89c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:35.918919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687c24b-5af3-4beb-b5f4-23dd4e5023c7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:35.921400 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 305/615] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:35.922495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb87c-af17-4637-9355-8c3304d7c89c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.955034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb878872-98e7-4e31-8d59-574a371fca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:35.955638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878872-98e7-4e31-8d59-574a371fca47 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:35.955928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878872-98e7-4e31-8d59-574a371fca47 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:35.956186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878872-98e7-4e31-8d59-574a371fca47 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:35.956801 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb878872-98e7-4e31-8d59-574a371fca47 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:35.957725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb878872-98e7-4e31-8d59-574a371fca47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:35.958449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb878872-98e7-4e31-8d59-574a371fca47 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:35.963107 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 311/616] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Oct 24 08:35:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:35.967876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb878872-98e7-4e31-8d59-574a371fca47 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:35.999028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:36.028408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fb87c-af17-4637-9355-8c3304d7c89c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.053262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:36.071876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf37d-5f6e-4444-8335-76816f1f46e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.072946 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf37d-5f6e-4444-8335-76816f1f46e3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.073479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf37d-5f6e-4444-8335-76816f1f46e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.073811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cf37d-5f6e-4444-8335-76816f1f46e3 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.074526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cf37d-5f6e-4444-8335-76816f1f46e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:36.074922 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cf37d-5f6e-4444-8335-76816f1f46e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.075402 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb878872-98e7-4e31-8d59-574a371fca47 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.076452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cf37d-5f6e-4444-8335-76816f1f46e3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.093275 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 306/617] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:36.093275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf37d-5f6e-4444-8335-76816f1f46e3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.116524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0add48f-8745-4895-b486-f51a04a667d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.117237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0add48f-8745-4895-b486-f51a04a667d4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.117577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0add48f-8745-4895-b486-f51a04a667d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.117913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0add48f-8745-4895-b486-f51a04a667d4 None admin] PATH_INFO: `/v3/services/network` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.118490 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0add48f-8745-4895-b486-f51a04a667d4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.119894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0add48f-8745-4895-b486-f51a04a667d4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c0add48f-8745-4895-b486-f51a04a667d4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:36.131391 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:36.132583 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 08:35:36.133642 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:36.136203 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 312/618] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:35 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:36.136797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0add48f-8745-4895-b486-f51a04a667d4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.203051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cf37d-5f6e-4444-8335-76816f1f46e3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.233246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:36.240207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0add48f-8745-4895-b486-f51a04a667d4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.247124 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdec8a0-29bd-40b5-9e93-68aa4fc1ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.248142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdec8a0-29bd-40b5-9e93-68aa4fc1ecff None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.248441 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdec8a0-29bd-40b5-9e93-68aa4fc1ecff None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.248712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdec8a0-29bd-40b5-9e93-68aa4fc1ecff None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.257686 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdec8a0-29bd-40b5-9e93-68aa4fc1ecff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.259166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdec8a0-29bd-40b5-9e93-68aa4fc1ecff None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.273400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329060ef-e541-4c83-be49-f4975b236adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-4bdec8a0-29bd-40b5-9e93-68aa4fc1ecff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:36.273569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:36.274429 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:36.275381 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:36.275381 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:36.275381 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:36.275381 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:36.275381 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:36.275381 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:36.275381 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:36.275381 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:36.275834 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329060ef-e541-4c83-be49-f4975b236adc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.275834 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329060ef-e541-4c83-be49-f4975b236adc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.275834 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329060ef-e541-4c83-be49-f4975b236adc None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.276012 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 307/619] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:36.276012 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdec8a0-29bd-40b5-9e93-68aa4fc1ecff None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.279972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329060ef-e541-4c83-be49-f4975b236adc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:36.280284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329060ef-e541-4c83-be49-f4975b236adc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.281661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329060ef-e541-4c83-be49-f4975b236adc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.287376 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 313/620] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Oct 24 08:35:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:36.288158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329060ef-e541-4c83-be49-f4975b236adc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.331384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:36.392470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329060ef-e541-4c83-be49-f4975b236adc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.392989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdec8a0-29bd-40b5-9e93-68aa4fc1ecff None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.425619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92c188-d081-4679-852d-b1416b6f7e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.426245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92c188-d081-4679-852d-b1416b6f7e0f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.426517 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92c188-d081-4679-852d-b1416b6f7e0f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.426807 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92c188-d081-4679-852d-b1416b6f7e0f None admin] PATH_INFO: `/v3/users/swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.427144 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23106d44-45a5-4f33-98a6-a42e789d5240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.427746 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23106d44-45a5-4f33-98a6-a42e789d5240 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.428024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23106d44-45a5-4f33-98a6-a42e789d5240 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.428295 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23106d44-45a5-4f33-98a6-a42e789d5240 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.428922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23106d44-45a5-4f33-98a6-a42e789d5240 None admin] RBAC: Adding query filter params (name=network) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:36.429225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23106d44-45a5-4f33-98a6-a42e789d5240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.430590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23106d44-45a5-4f33-98a6-a42e789d5240 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.434481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92c188-d081-4679-852d-b1416b6f7e0f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.439921 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 314/621] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:36.440046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92c188-d081-4679-852d-b1416b6f7e0f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.440123 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23106d44-45a5-4f33-98a6-a42e789d5240 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-0a92c188-d081-4679-852d-b1416b6f7e0f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:36.453061 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:36.454050 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:36.455058 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 308/622] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:35:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:36.455669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92c188-d081-4679-852d-b1416b6f7e0f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.469108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:36.547589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23106d44-45a5-4f33-98a6-a42e789d5240 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.560121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:36.563955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92c188-d081-4679-852d-b1416b6f7e0f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.581956 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36e9bb-7e6a-4714-b2a5-ffb5e421b0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.582566 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e9bb-7e6a-4714-b2a5-ffb5e421b0b3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.582876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e9bb-7e6a-4714-b2a5-ffb5e421b0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.583150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e9bb-7e6a-4714-b2a5-ffb5e421b0b3 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.583840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36e9bb-7e6a-4714-b2a5-ffb5e421b0b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:36.584145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36e9bb-7e6a-4714-b2a5-ffb5e421b0b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.585579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36e9bb-7e6a-4714-b2a5-ffb5e421b0b3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.597892 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 315/623] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:36.598701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36e9bb-7e6a-4714-b2a5-ffb5e421b0b3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.608294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf36a4-4eb6-43c4-95cd-cd3ad50f7989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.609077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf36a4-4eb6-43c4-95cd-cd3ad50f7989 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.609447 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf36a4-4eb6-43c4-95cd-cd3ad50f7989 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.609804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf36a4-4eb6-43c4-95cd-cd3ad50f7989 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.610616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf36a4-4eb6-43c4-95cd-cd3ad50f7989 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.612427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf36a4-4eb6-43c4-95cd-cd3ad50f7989 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.641117 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 309/624] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:35:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:36.641940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf36a4-4eb6-43c4-95cd-cd3ad50f7989 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.690079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:36.714494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36e9bb-7e6a-4714-b2a5-ffb5e421b0b3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.761946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5101fe-a06f-4914-b7fc-e91e44d2a933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.762676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5101fe-a06f-4914-b7fc-e91e44d2a933 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.763149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5101fe-a06f-4914-b7fc-e91e44d2a933 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.763419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5101fe-a06f-4914-b7fc-e91e44d2a933 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.764051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5101fe-a06f-4914-b7fc-e91e44d2a933 None admin] RBAC: Adding query filter params (type=network) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:36.764350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5101fe-a06f-4914-b7fc-e91e44d2a933 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.765754 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5101fe-a06f-4914-b7fc-e91e44d2a933 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.776017 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 316/625] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:36.776017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5101fe-a06f-4914-b7fc-e91e44d2a933 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.785893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf36a4-4eb6-43c4-95cd-cd3ad50f7989 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.825118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f54009-da21-481b-9872-cfab24643b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.825764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f54009-da21-481b-9872-cfab24643b4b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.825962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f54009-da21-481b-9872-cfab24643b4b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.826289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f54009-da21-481b-9872-cfab24643b4b None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.827107 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f54009-da21-481b-9872-cfab24643b4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:36.827460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f54009-da21-481b-9872-cfab24643b4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.829100 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f54009-da21-481b-9872-cfab24643b4b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.853620 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 310/626] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Oct 24 08:35:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:36.853620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f54009-da21-481b-9872-cfab24643b4b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.869431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:36.896666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5101fe-a06f-4914-b7fc-e91e44d2a933 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:36.938944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785fea6-2e11-4804-b76f-87eabc0f29a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:36.939557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785fea6-2e11-4804-b76f-87eabc0f29a7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:36.939830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785fea6-2e11-4804-b76f-87eabc0f29a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:36.940098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785fea6-2e11-4804-b76f-87eabc0f29a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:36.940841 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785fea6-2e11-4804-b76f-87eabc0f29a7 None admin] RBAC: Adding query filter params (role.id=cb7d27022ded4ea2afd3d5ebaefbf429, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=771fc0da7a8749f0b80ce0630951500a) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:36.941217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785fea6-2e11-4804-b76f-87eabc0f29a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:36.942924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785fea6-2e11-4804-b76f-87eabc0f29a7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:36.949873 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 317/627] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:35:36 2022] GET /identity/v3/role_assignments?user.id=771fc0da7a8749f0b80ce0630951500a&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=cb7d27022ded4ea2afd3d5ebaefbf429 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:36.951280 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785fea6-2e11-4804-b76f-87eabc0f29a7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:36.961955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:37.012911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f54009-da21-481b-9872-cfab24643b4b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:37.064832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785fea6-2e11-4804-b76f-87eabc0f29a7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:37.087626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67fec8-ad0a-4d79-b9e3-e31f7d75b3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:37.088196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67fec8-ad0a-4d79-b9e3-e31f7d75b3f1 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:37.088475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67fec8-ad0a-4d79-b9e3-e31f7d75b3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:37.088743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67fec8-ad0a-4d79-b9e3-e31f7d75b3f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:37.089514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67fec8-ad0a-4d79-b9e3-e31f7d75b3f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:37.090935 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67fec8-ad0a-4d79-b9e3-e31f7d75b3f1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:37.102275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d661734-6bea-4a70-ba24-0d4d40ba75fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:37.102985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d661734-6bea-4a70-ba24-0d4d40ba75fb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:37.103214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d661734-6bea-4a70-ba24-0d4d40ba75fb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:37.103482 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d661734-6bea-4a70-ba24-0d4d40ba75fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:37.164696 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 311/628] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:35:36 2022] POST /identity/v3/endpoints => generated 337 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:37.182479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d661734-6bea-4a70-ba24-0d4d40ba75fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:37.184209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d661734-6bea-4a70-ba24-0d4d40ba75fb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:37.199216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:37.298094 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 318/629] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:35:37.304571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67fec8-ad0a-4d79-b9e3-e31f7d75b3f1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:37.394896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:37.441876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67fec8-ad0a-4d79-b9e3-e31f7d75b3f1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:37.478860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb916882-3355-44f9-a814-af14b535a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:37.479880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb916882-3355-44f9-a814-af14b535a488 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:37.480293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb916882-3355-44f9-a814-af14b535a488 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:37.480694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb916882-3355-44f9-a814-af14b535a488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:37.493537 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb916882-3355-44f9-a814-af14b535a488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:37.495384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb916882-3355-44f9-a814-af14b535a488 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-fb916882-3355-44f9-a814-af14b535a488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:37.510094 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:37.511569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:37.512665 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:37.512665 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:37.512665 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:37.514397 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 312/630] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:37.521042 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d661734-6bea-4a70-ba24-0d4d40ba75fb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:37.540948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4297bc-92fc-4040-bf88-4e73b5d2df79 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:37.541472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4297bc-92fc-4040-bf88-4e73b5d2df79 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:37.541745 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4297bc-92fc-4040-bf88-4e73b5d2df79 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:37.543433 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 313/631] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:35:37.553694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53ea4e-62c0-4adc-ae9e-f8fabd6da1d5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:37.553694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53ea4e-62c0-4adc-ae9e-f8fabd6da1d5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:37.553939 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53ea4e-62c0-4adc-ae9e-f8fabd6da1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:37.577742 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ba53ea4e-62c0-4adc-ae9e-f8fabd6da1d5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:37.579322 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ba53ea4e-62c0-4adc-ae9e-f8fabd6da1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:37.655516 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 314/632] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:37.656520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d661734-6bea-4a70-ba24-0d4d40ba75fb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:37.667430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:37.699599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44be52a-60b6-49ff-80b3-9175ffd91579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:37.700227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44be52a-60b6-49ff-80b3-9175ffd91579 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:37.700504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44be52a-60b6-49ff-80b3-9175ffd91579 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:37.700773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44be52a-60b6-49ff-80b3-9175ffd91579 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:37.701439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44be52a-60b6-49ff-80b3-9175ffd91579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:37.701741 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44be52a-60b6-49ff-80b3-9175ffd91579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:37.703212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44be52a-60b6-49ff-80b3-9175ffd91579 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:37.720234 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 319/633] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:37.722850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53ea4e-62c0-4adc-ae9e-f8fabd6da1d5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:37.775253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:37.825965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53ea4e-62c0-4adc-ae9e-f8fabd6da1d5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:37.859625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d06d4f-b7e7-456e-8dcc-04827d55c757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:37.860268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d06d4f-b7e7-456e-8dcc-04827d55c757 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:37.860549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d06d4f-b7e7-456e-8dcc-04827d55c757 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:37.860814 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d06d4f-b7e7-456e-8dcc-04827d55c757 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:37.867494 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d06d4f-b7e7-456e-8dcc-04827d55c757 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:37.869006 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d06d4f-b7e7-456e-8dcc-04827d55c757 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-30d06d4f-b7e7-456e-8dcc-04827d55c757 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:37.880038 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:37.880932 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:37.881874 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:37.881874 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:37.881874 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:37.881874 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:37.881874 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:37.881874 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:37.881874 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:37.881874 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:37.886055 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 315/634] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:37.887491 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44be52a-60b6-49ff-80b3-9175ffd91579 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:37.942426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d06d4f-b7e7-456e-8dcc-04827d55c757 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:38.004609 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44be52a-60b6-49ff-80b3-9175ffd91579 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:38.041527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183b3b8-1bbd-43cb-932d-b9f334a2ed78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:38.042199 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183b3b8-1bbd-43cb-932d-b9f334a2ed78 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:38.042499 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183b3b8-1bbd-43cb-932d-b9f334a2ed78 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:38.042885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183b3b8-1bbd-43cb-932d-b9f334a2ed78 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:38.043582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183b3b8-1bbd-43cb-932d-b9f334a2ed78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:38.043886 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183b3b8-1bbd-43cb-932d-b9f334a2ed78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:38.045445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183b3b8-1bbd-43cb-932d-b9f334a2ed78 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:38.065789 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 320/635] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:38.067263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183b3b8-1bbd-43cb-932d-b9f334a2ed78 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:38.068668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d06d4f-b7e7-456e-8dcc-04827d55c757 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:38.081199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:38.096752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:38.120242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bbf70-0da6-43f7-87fd-5fc6aa0675f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:38.121134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bbf70-0da6-43f7-87fd-5fc6aa0675f7 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:38.121423 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bbf70-0da6-43f7-87fd-5fc6aa0675f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:38.121704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bbf70-0da6-43f7-87fd-5fc6aa0675f7 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:38.122636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bbf70-0da6-43f7-87fd-5fc6aa0675f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:38.124647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bbf70-0da6-43f7-87fd-5fc6aa0675f7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:38.146425 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 316/636] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:35:37 2022] POST /identity/v3/services => generated 269 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:35:38.187893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183b3b8-1bbd-43cb-932d-b9f334a2ed78 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:38.235325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45867b11-3d45-4297-963d-05f7c2c36ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:38.235918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45867b11-3d45-4297-963d-05f7c2c36ceb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:38.236196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45867b11-3d45-4297-963d-05f7c2c36ceb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:38.236465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45867b11-3d45-4297-963d-05f7c2c36ceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:38.237203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45867b11-3d45-4297-963d-05f7c2c36ceb None admin] RBAC: Adding query filter params (role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=dbf288ceecac4fec8b2161c55a61a8ba) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:38.237508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45867b11-3d45-4297-963d-05f7c2c36ceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:38.239015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45867b11-3d45-4297-963d-05f7c2c36ceb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:38.251061 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 321/637] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:35:38 2022] GET /identity/v3/role_assignments?user.id=dbf288ceecac4fec8b2161c55a61a8ba&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:39.006448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9363ce-ccbe-49f5-91a8-b44109b27cee None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:39.007013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9363ce-ccbe-49f5-91a8-b44109b27cee None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:39.007013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9363ce-ccbe-49f5-91a8-b44109b27cee None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:39.008375 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 317/638] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:39.013822 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68debd16-5c21-4967-a1f7-b66154a73bb3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:39.014243 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68debd16-5c21-4967-a1f7-b66154a73bb3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:39.014414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68debd16-5c21-4967-a1f7-b66154a73bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:39.033072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-68debd16-5c21-4967-a1f7-b66154a73bb3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:39.034277 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-68debd16-5c21-4967-a1f7-b66154a73bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:39.105601 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 322/639] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:39 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:39.372475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9363ce-ccbe-49f5-91a8-b44109b27cee None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:39.489262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9363ce-ccbe-49f5-91a8-b44109b27cee None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:39.523477 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67e52-b0e9-47b4-b8fb-c94e5ff780af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:39.524118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67e52-b0e9-47b4-b8fb-c94e5ff780af None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:39.524402 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67e52-b0e9-47b4-b8fb-c94e5ff780af None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:39.524679 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67e52-b0e9-47b4-b8fb-c94e5ff780af None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:39.525253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff67e52-b0e9-47b4-b8fb-c94e5ff780af None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:39.526701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff67e52-b0e9-47b4-b8fb-c94e5ff780af None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-5ff67e52-b0e9-47b4-b8fb-c94e5ff780af None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:39.535632 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:39.537017 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:39.537860 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 318/640] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:39.543417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68debd16-5c21-4967-a1f7-b66154a73bb3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:39.561368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:39.589442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0429676-5dca-4492-8365-002fa4cdabbc None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:39.589758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0429676-5dca-4492-8365-002fa4cdabbc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:39.590005 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0429676-5dca-4492-8365-002fa4cdabbc None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:39.595303 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 319/641] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:35:39.596551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364559e2-d2a4-4a79-9f96-d907fc7859a8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:39.596840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364559e2-d2a4-4a79-9f96-d907fc7859a8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:39.597111 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364559e2-d2a4-4a79-9f96-d907fc7859a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:39.618855 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-364559e2-d2a4-4a79-9f96-d907fc7859a8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:39.620356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-364559e2-d2a4-4a79-9f96-d907fc7859a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:39.655295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:39.657056 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68debd16-5c21-4967-a1f7-b66154a73bb3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:39.707708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56e993-271f-4e51-8956-59c70e04de35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:39.708319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56e993-271f-4e51-8956-59c70e04de35 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:39.708598 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56e993-271f-4e51-8956-59c70e04de35 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:39.708925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56e993-271f-4e51-8956-59c70e04de35 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:39.709551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56e993-271f-4e51-8956-59c70e04de35 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:39.709757 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 320/642] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:39 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:39.709859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56e993-271f-4e51-8956-59c70e04de35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:39.715321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56e993-271f-4e51-8956-59c70e04de35 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:39.720799 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 323/643] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:39.726288 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364559e2-d2a4-4a79-9f96-d907fc7859a8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:39.827099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364559e2-d2a4-4a79-9f96-d907fc7859a8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:39.860801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e4baf-d080-4a30-8fe5-07a7cf7cdc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:39.861419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e4baf-d080-4a30-8fe5-07a7cf7cdc09 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:39.861699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e4baf-d080-4a30-8fe5-07a7cf7cdc09 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:39.861971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e4baf-d080-4a30-8fe5-07a7cf7cdc09 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:39.862603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e4baf-d080-4a30-8fe5-07a7cf7cdc09 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:39.862932 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e4baf-d080-4a30-8fe5-07a7cf7cdc09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:39.864446 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e4baf-d080-4a30-8fe5-07a7cf7cdc09 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:39.869343 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 321/644] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:40.021160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56e993-271f-4e51-8956-59c70e04de35 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:40.112368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:40.132295 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56e993-271f-4e51-8956-59c70e04de35 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:40.166867 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dae27-98dc-4c30-8034-50a7cef228d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:40.167976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dae27-98dc-4c30-8034-50a7cef228d1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.168450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dae27-98dc-4c30-8034-50a7cef228d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.168911 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dae27-98dc-4c30-8034-50a7cef228d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.179290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dae27-98dc-4c30-8034-50a7cef228d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0a4dae27-98dc-4c30-8034-50a7cef228d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:40.184323 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:40.185289 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 08:35:40.186149 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:40.193414 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 324/645] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Oct 24 08:35:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:40.195174 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e4baf-d080-4a30-8fe5-07a7cf7cdc09 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:40.226261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:40.314751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e4baf-d080-4a30-8fe5-07a7cf7cdc09 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:40.348877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c939a1-6a0f-4dc3-bb6d-8e4ea5e4a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:40.349542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c939a1-6a0f-4dc3-bb6d-8e4ea5e4a12d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.349818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c939a1-6a0f-4dc3-bb6d-8e4ea5e4a12d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.350085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c939a1-6a0f-4dc3-bb6d-8e4ea5e4a12d None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.350761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c939a1-6a0f-4dc3-bb6d-8e4ea5e4a12d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:40.351066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c939a1-6a0f-4dc3-bb6d-8e4ea5e4a12d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:40.352434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c939a1-6a0f-4dc3-bb6d-8e4ea5e4a12d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:40.360944 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 322/646] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Oct 24 08:35:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:40.366932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dae27-98dc-4c30-8034-50a7cef228d1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:40.380554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee986fd-ffca-4350-a68f-f40709358196 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.380910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee986fd-ffca-4350-a68f-f40709358196 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.381115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee986fd-ffca-4350-a68f-f40709358196 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.383854 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 323/647] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:35:40.388994 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab0947-3b4b-4b33-99b8-1be3d89af50e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.389384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab0947-3b4b-4b33-99b8-1be3d89af50e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.389761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab0947-3b4b-4b33-99b8-1be3d89af50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.409216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-05ab0947-3b4b-4b33-99b8-1be3d89af50e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:40.410579 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-05ab0947-3b4b-4b33-99b8-1be3d89af50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:40.479715 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dae27-98dc-4c30-8034-50a7cef228d1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:40.487410 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 324/648] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:40 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:40.500772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89ce89-330d-4569-bb0a-b41e13de66c7 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.501182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89ce89-330d-4569-bb0a-b41e13de66c7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.501552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89ce89-330d-4569-bb0a-b41e13de66c7 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.503271 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 325/649] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:40.508002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092748e-d859-4264-ad1c-8f66949e2d55 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.508287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092748e-d859-4264-ad1c-8f66949e2d55 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.508560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092748e-d859-4264-ad1c-8f66949e2d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.527163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11e7ea-7a60-40d2-9149-f6c88d12edd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:40.527935 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2092748e-d859-4264-ad1c-8f66949e2d55 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:40.528368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11e7ea-7a60-40d2-9149-f6c88d12edd5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.528858 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11e7ea-7a60-40d2-9149-f6c88d12edd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.529117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2092748e-d859-4264-ad1c-8f66949e2d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:40.529523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11e7ea-7a60-40d2-9149-f6c88d12edd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.541254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:40.544075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:40.546229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11e7ea-7a60-40d2-9149-f6c88d12edd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:40.548707 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11e7ea-7a60-40d2-9149-f6c88d12edd5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-da11e7ea-7a60-40d2-9149-f6c88d12edd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:40.571751 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:40.572769 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:40.573747 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:40.573747 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:40.573747 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:40.580754 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 325/650] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:40.584078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11e7ea-7a60-40d2-9149-f6c88d12edd5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:40.619116 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 326/651] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:40 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:40.690938 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11e7ea-7a60-40d2-9149-f6c88d12edd5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:40.724403 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c0116-2832-4f7b-a907-25e7ee70b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:40.725360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c0116-2832-4f7b-a907-25e7ee70b0af None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.725861 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c0116-2832-4f7b-a907-25e7ee70b0af None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.726367 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c0116-2832-4f7b-a907-25e7ee70b0af None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.727259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c0116-2832-4f7b-a907-25e7ee70b0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:40.727756 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c0116-2832-4f7b-a907-25e7ee70b0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:40.729338 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c0116-2832-4f7b-a907-25e7ee70b0af None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:40.746152 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 326/652] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:40.747798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092748e-d859-4264-ad1c-8f66949e2d55 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:40.751329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c0116-2832-4f7b-a907-25e7ee70b0af None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:40.806900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:40.856513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c0116-2832-4f7b-a907-25e7ee70b0af None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:40.876915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092748e-d859-4264-ad1c-8f66949e2d55 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:40.891871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dcc51-d494-4606-9d90-9a5e47dbe351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:40.892449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dcc51-d494-4606-9d90-9a5e47dbe351 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.892726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dcc51-d494-4606-9d90-9a5e47dbe351 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.892996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dcc51-d494-4606-9d90-9a5e47dbe351 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.893579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467dcc51-d494-4606-9d90-9a5e47dbe351 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:40.894985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467dcc51-d494-4606-9d90-9a5e47dbe351 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-467dcc51-d494-4606-9d90-9a5e47dbe351 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:40.903929 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:40.905071 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:40.906114 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 327/653] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Oct 24 08:35:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:35:40.906997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dcc51-d494-4606-9d90-9a5e47dbe351 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:40.920160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4d5a1-76cd-4df1-be82-3ec88ce6fa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:40.921007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4d5a1-76cd-4df1-be82-3ec88ce6fa20 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:40.921433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4d5a1-76cd-4df1-be82-3ec88ce6fa20 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:40.921958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4d5a1-76cd-4df1-be82-3ec88ce6fa20 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:40.929491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4d5a1-76cd-4df1-be82-3ec88ce6fa20 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:40.931233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4d5a1-76cd-4df1-be82-3ec88ce6fa20 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-77b4d5a1-76cd-4df1-be82-3ec88ce6fa20 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:40.943498 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:40.944628 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 08:35:40.945711 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:40.947524 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 327/654] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Oct 24 08:35:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:40.948469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4d5a1-76cd-4df1-be82-3ec88ce6fa20 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:40.961239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:41.013138 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dcc51-d494-4606-9d90-9a5e47dbe351 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.029162 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:41.048127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc565339-a276-4d4b-a5fa-d95eab435ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.048751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc565339-a276-4d4b-a5fa-d95eab435ba3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.049032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc565339-a276-4d4b-a5fa-d95eab435ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.049311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc565339-a276-4d4b-a5fa-d95eab435ba3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.049885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc565339-a276-4d4b-a5fa-d95eab435ba3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.051295 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc565339-a276-4d4b-a5fa-d95eab435ba3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.058869 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4d5a1-76cd-4df1-be82-3ec88ce6fa20 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-dc565339-a276-4d4b-a5fa-d95eab435ba3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:41.060059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:41.061043 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:41.061879 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 328/655] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:41.063145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc565339-a276-4d4b-a5fa-d95eab435ba3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:41.104612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ac984-55f1-419b-ba86-d3f9723c9c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.105375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ac984-55f1-419b-ba86-d3f9723c9c40 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.105658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ac984-55f1-419b-ba86-d3f9723c9c40 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.105941 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ac984-55f1-419b-ba86-d3f9723c9c40 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.106572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ac984-55f1-419b-ba86-d3f9723c9c40 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:41.106910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ac984-55f1-419b-ba86-d3f9723c9c40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.108295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ac984-55f1-419b-ba86-d3f9723c9c40 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.117794 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 328/656] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:41.117794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ac984-55f1-419b-ba86-d3f9723c9c40 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:41.171423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc565339-a276-4d4b-a5fa-d95eab435ba3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.189321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:41.212058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a1fa3-d093-445e-9455-918f3edd3ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.213078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a1fa3-d093-445e-9455-918f3edd3ffb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.213843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a1fa3-d093-445e-9455-918f3edd3ffb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.214405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a1fa3-d093-445e-9455-918f3edd3ffb None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.215512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a1fa3-d093-445e-9455-918f3edd3ffb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:41.215969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a1fa3-d093-445e-9455-918f3edd3ffb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.217537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a1fa3-d093-445e-9455-918f3edd3ffb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.219366 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ac984-55f1-419b-ba86-d3f9723c9c40 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.237987 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 329/657] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:41.239030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a1fa3-d093-445e-9455-918f3edd3ffb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:41.259098 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aee53f-0022-44ca-9906-8d1f2139049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.259721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aee53f-0022-44ca-9906-8d1f2139049a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.259996 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aee53f-0022-44ca-9906-8d1f2139049a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.260264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aee53f-0022-44ca-9906-8d1f2139049a None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.260984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aee53f-0022-44ca-9906-8d1f2139049a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:41.261511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aee53f-0022-44ca-9906-8d1f2139049a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.262649 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aee53f-0022-44ca-9906-8d1f2139049a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.268739 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 329/658] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:41.268739 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aee53f-0022-44ca-9906-8d1f2139049a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:41.287831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:41.350006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a1fa3-d093-445e-9455-918f3edd3ffb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.372456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aee53f-0022-44ca-9906-8d1f2139049a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.386073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9861f-322a-416f-8e31-e61f9bc34594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.386776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9861f-322a-416f-8e31-e61f9bc34594 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.387113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9861f-322a-416f-8e31-e61f9bc34594 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.387402 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9861f-322a-416f-8e31-e61f9bc34594 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.387993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9861f-322a-416f-8e31-e61f9bc34594 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:41.388301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9861f-322a-416f-8e31-e61f9bc34594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.389721 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9861f-322a-416f-8e31-e61f9bc34594 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.394818 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 330/659] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:41.395593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9861f-322a-416f-8e31-e61f9bc34594 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:41.410705 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01af8de-9dd6-43af-a2b6-1cf531a0efe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.411715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01af8de-9dd6-43af-a2b6-1cf531a0efe0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.411994 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01af8de-9dd6-43af-a2b6-1cf531a0efe0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.412267 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01af8de-9dd6-43af-a2b6-1cf531a0efe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.414951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:41.423961 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01af8de-9dd6-43af-a2b6-1cf531a0efe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.425334 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01af8de-9dd6-43af-a2b6-1cf531a0efe0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f01af8de-9dd6-43af-a2b6-1cf531a0efe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:41.439603 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:41.440761 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:41.441867 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:41.441867 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:41.441867 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:41.441867 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 330/660] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:41.441867 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01af8de-9dd6-43af-a2b6-1cf531a0efe0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:41.517213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9861f-322a-416f-8e31-e61f9bc34594 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.518946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:41.552027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01af8de-9dd6-43af-a2b6-1cf531a0efe0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.554250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83cd10-803c-4720-8ccc-a4c747b5c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.555205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83cd10-803c-4720-8ccc-a4c747b5c9cf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.555205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83cd10-803c-4720-8ccc-a4c747b5c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.555506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83cd10-803c-4720-8ccc-a4c747b5c9cf None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.556061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83cd10-803c-4720-8ccc-a4c747b5c9cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:41.556366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83cd10-803c-4720-8ccc-a4c747b5c9cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.557806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83cd10-803c-4720-8ccc-a4c747b5c9cf None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.566818 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 331/661] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:41.566818 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83cd10-803c-4720-8ccc-a4c747b5c9cf None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:41.602098 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c0fce-12a2-4449-a453-5416b9b35234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.602769 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c0fce-12a2-4449-a453-5416b9b35234 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.603037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c0fce-12a2-4449-a453-5416b9b35234 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.603304 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c0fce-12a2-4449-a453-5416b9b35234 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.604026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c0fce-12a2-4449-a453-5416b9b35234 None admin] RBAC: Adding query filter params (interface=public, service_id=d4826b8235ed4fc7ab8c187d660200c2, region_id=RegionOne) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:41.604331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c0fce-12a2-4449-a453-5416b9b35234 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.605802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c0fce-12a2-4449-a453-5416b9b35234 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.616763 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 331/662] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/endpoints?service_id=d4826b8235ed4fc7ab8c187d660200c2&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:41.617670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c0fce-12a2-4449-a453-5416b9b35234 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:41.685226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:41.697518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83cd10-803c-4720-8ccc-a4c747b5c9cf None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.742598 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebef3b-df86-45b7-81eb-a5cf210a7878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.744296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebef3b-df86-45b7-81eb-a5cf210a7878 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.744802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebef3b-df86-45b7-81eb-a5cf210a7878 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.745352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebef3b-df86-45b7-81eb-a5cf210a7878 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.746896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebef3b-df86-45b7-81eb-a5cf210a7878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:41.747529 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebef3b-df86-45b7-81eb-a5cf210a7878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.749240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebef3b-df86-45b7-81eb-a5cf210a7878 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.753738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c0fce-12a2-4449-a453-5416b9b35234 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.767524 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 332/663] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:41.773415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebef3b-df86-45b7-81eb-a5cf210a7878 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:41.797702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51495815-ae3a-49d1-8e3c-b998f644fa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.798326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51495815-ae3a-49d1-8e3c-b998f644fa17 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.798996 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51495815-ae3a-49d1-8e3c-b998f644fa17 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.798996 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51495815-ae3a-49d1-8e3c-b998f644fa17 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.799513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51495815-ae3a-49d1-8e3c-b998f644fa17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.800893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51495815-ae3a-49d1-8e3c-b998f644fa17 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.807093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:41.808775 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 332/664] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/services => generated 1029 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:35:41.894765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebef3b-df86-45b7-81eb-a5cf210a7878 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:41.933507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660d8cb-fca9-4491-90b6-251cb136dca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:41.934223 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660d8cb-fca9-4491-90b6-251cb136dca9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:41.934507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660d8cb-fca9-4491-90b6-251cb136dca9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:41.934816 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660d8cb-fca9-4491-90b6-251cb136dca9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:41.943546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660d8cb-fca9-4491-90b6-251cb136dca9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:41.944993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660d8cb-fca9-4491-90b6-251cb136dca9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9660d8cb-fca9-4491-90b6-251cb136dca9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:41.959059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:41.960042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:41.960921 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:41.960921 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:41.960921 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:41.960921 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:41.960921 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:41.960921 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:41.960921 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:41.960921 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:41.960921 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 333/665] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:41.967115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51495815-ae3a-49d1-8e3c-b998f644fa17 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:42.069623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10909fba-1d3c-4b01-ab6e-9934a6fba19a None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:42.070177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10909fba-1d3c-4b01-ab6e-9934a6fba19a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:42.070177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10909fba-1d3c-4b01-ab6e-9934a6fba19a None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:42.072866 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 334/666] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:42.077098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dade86c-4f19-48b1-aee7-91ff8fed308c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:42.077383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dade86c-4f19-48b1-aee7-91ff8fed308c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:42.077655 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dade86c-4f19-48b1-aee7-91ff8fed308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:42.081646 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51495815-ae3a-49d1-8e3c-b998f644fa17 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:42.097828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5dade86c-4f19-48b1-aee7-91ff8fed308c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:42.099410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5dade86c-4f19-48b1-aee7-91ff8fed308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:42.100129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:42.119324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64a6ed-728b-4d03-8678-78ef9afc0a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:42.119943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64a6ed-728b-4d03-8678-78ef9afc0a4b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:42.120264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64a6ed-728b-4d03-8678-78ef9afc0a4b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:42.120537 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64a6ed-728b-4d03-8678-78ef9afc0a4b None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:42.121260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64a6ed-728b-4d03-8678-78ef9afc0a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:42.121564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64a6ed-728b-4d03-8678-78ef9afc0a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:42.123030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64a6ed-728b-4d03-8678-78ef9afc0a4b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:42.135850 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 333/667] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:42.141438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64a6ed-728b-4d03-8678-78ef9afc0a4b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:42.186841 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 335/668] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:42 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:42.254389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64a6ed-728b-4d03-8678-78ef9afc0a4b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:42.290574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af6124-b50d-4dbd-b73c-8b86131b1af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:42.291283 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af6124-b50d-4dbd-b73c-8b86131b1af3 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:42.291563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af6124-b50d-4dbd-b73c-8b86131b1af3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:42.291832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af6124-b50d-4dbd-b73c-8b86131b1af3 None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/771fc0da7a8749f0b80ce0630951500a/roles/1c0b192e409a435a98f939e77554596d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:42.294032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af6124-b50d-4dbd-b73c-8b86131b1af3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=771fc0da7a8749f0b80ce0630951500a, role_id=1c0b192e409a435a98f939e77554596d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:42.295483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af6124-b50d-4dbd-b73c-8b86131b1af3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:42.312547 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 334/669] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:35:42 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/771fc0da7a8749f0b80ce0630951500a/roles/1c0b192e409a435a98f939e77554596d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:35:42.453247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dade86c-4f19-48b1-aee7-91ff8fed308c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:42.472739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:42.562321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dade86c-4f19-48b1-aee7-91ff8fed308c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:42.597629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df064e1d-4ae8-43e2-97f2-249e6b44385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:42.598247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df064e1d-4ae8-43e2-97f2-249e6b44385e None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:42.598522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df064e1d-4ae8-43e2-97f2-249e6b44385e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:42.598818 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df064e1d-4ae8-43e2-97f2-249e6b44385e None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:42.599479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df064e1d-4ae8-43e2-97f2-249e6b44385e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:42.600842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df064e1d-4ae8-43e2-97f2-249e6b44385e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:42.622926 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 336/670] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:35:42 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:43.785819 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fea840-2218-4ba4-a52b-5145e62f1961 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:43.786772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fea840-2218-4ba4-a52b-5145e62f1961 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:43.786772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fea840-2218-4ba4-a52b-5145e62f1961 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:43.788365 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 335/671] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:43.793937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb08460-86a8-45a5-a11a-728edc118a66 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:43.794338 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb08460-86a8-45a5-a11a-728edc118a66 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:43.794535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb08460-86a8-45a5-a11a-728edc118a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:43.813700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6fb08460-86a8-45a5-a11a-728edc118a66 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:43.814930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6fb08460-86a8-45a5-a11a-728edc118a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:43.886619 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 337/672] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:44.074863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd334841-e281-4af7-8c71-ec6d437c5260 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.075085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd334841-e281-4af7-8c71-ec6d437c5260 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.075358 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd334841-e281-4af7-8c71-ec6d437c5260 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.076931 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 336/673] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:44.084234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5700f-3f5b-4b5d-86fa-f01576f2c8c6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.084628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5700f-3f5b-4b5d-86fa-f01576f2c8c6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.085000 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5700f-3f5b-4b5d-86fa-f01576f2c8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.091721 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:44.107667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c5c5700f-3f5b-4b5d-86fa-f01576f2c8c6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:44.108858 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c5c5700f-3f5b-4b5d-86fa-f01576f2c8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:44.151729 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd334841-e281-4af7-8c71-ec6d437c5260 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:44.184173 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 338/674] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:44.192801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:44.258967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd334841-e281-4af7-8c71-ec6d437c5260 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:44.294037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c9a4e-2df0-42c6-9c94-1c6d4ad02cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:44.294661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c9a4e-2df0-42c6-9c94-1c6d4ad02cdc None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.294974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c9a4e-2df0-42c6-9c94-1c6d4ad02cdc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.295373 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c9a4e-2df0-42c6-9c94-1c6d4ad02cdc None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.296037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c9a4e-2df0-42c6-9c94-1c6d4ad02cdc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:44.297421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c9a4e-2df0-42c6-9c94-1c6d4ad02cdc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:44.315238 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 337/675] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:35:44 2022] POST /identity/v3/services => generated 241 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:44.461947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5700f-3f5b-4b5d-86fa-f01576f2c8c6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:44.547744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f097f-e90d-48c9-8fb0-3391adfdfddb None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.548402 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f097f-e90d-48c9-8fb0-3391adfdfddb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.548906 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f097f-e90d-48c9-8fb0-3391adfdfddb None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.550709 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 338/676] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:35:44.556161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a4341-6628-4881-88ca-c51383bf955b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.556710 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a4341-6628-4881-88ca-c51383bf955b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.557199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a4341-6628-4881-88ca-c51383bf955b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.572188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5700f-3f5b-4b5d-86fa-f01576f2c8c6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:44.582625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b05a4341-6628-4881-88ca-c51383bf955b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:44.584869 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b05a4341-6628-4881-88ca-c51383bf955b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:44.607006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:44.624993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4d36b-9609-4851-97a4-06d33be38e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:44.625785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4d36b-9609-4851-97a4-06d33be38e85 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.626061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4d36b-9609-4851-97a4-06d33be38e85 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.626328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4d36b-9609-4851-97a4-06d33be38e85 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.626929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4d36b-9609-4851-97a4-06d33be38e85 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:44.628891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4d36b-9609-4851-97a4-06d33be38e85 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1bc4d36b-9609-4851-97a4-06d33be38e85 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:44.637722 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:44.638759 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:44.639680 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 339/677] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Oct 24 08:35:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:44.646314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4d36b-9609-4851-97a4-06d33be38e85 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:44.684779 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 339/678] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:44.762622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4d36b-9609-4851-97a4-06d33be38e85 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:44.797392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a3da1-dd25-4f08-84a2-82d164e0f028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:44.798059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a3da1-dd25-4f08-84a2-82d164e0f028 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.798300 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a3da1-dd25-4f08-84a2-82d164e0f028 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.798516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a3da1-dd25-4f08-84a2-82d164e0f028 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.799171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a3da1-dd25-4f08-84a2-82d164e0f028 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:44.799471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a3da1-dd25-4f08-84a2-82d164e0f028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:44.800824 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a3da1-dd25-4f08-84a2-82d164e0f028 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:44.806040 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 340/679] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:44.812086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a4341-6628-4881-88ca-c51383bf955b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:44.842920 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:44.844559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ed2cf-880a-4a80-bf53-91e036bcca38 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.844879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ed2cf-880a-4a80-bf53-91e036bcca38 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.845108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ed2cf-880a-4a80-bf53-91e036bcca38 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.846593 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 341/680] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:44.853065 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7272665-5318-42a1-b319-d686fc8edca4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.853339 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7272665-5318-42a1-b319-d686fc8edca4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.853610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7272665-5318-42a1-b319-d686fc8edca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.871989 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b7272665-5318-42a1-b319-d686fc8edca4 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:44.873496 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b7272665-5318-42a1-b319-d686fc8edca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:44.927231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:44.939766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a4341-6628-4881-88ca-c51383bf955b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:44.954738 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 342/681] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:44.965427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7272665-5318-42a1-b319-d686fc8edca4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:44.981545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97fc6e-dbe0-445c-a0ab-b4114985b912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:44.982433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97fc6e-dbe0-445c-a0ab-b4114985b912 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:44.982776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97fc6e-dbe0-445c-a0ab-b4114985b912 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:44.983060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97fc6e-dbe0-445c-a0ab-b4114985b912 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:44.983891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97fc6e-dbe0-445c-a0ab-b4114985b912 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:44.984464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97fc6e-dbe0-445c-a0ab-b4114985b912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:44.985972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97fc6e-dbe0-445c-a0ab-b4114985b912 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:44.992914 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 340/682] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:44.998544 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97fc6e-dbe0-445c-a0ab-b4114985b912 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.065733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7272665-5318-42a1-b319-d686fc8edca4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.092670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:45.100462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0eba69-fa4e-471b-8f39-1e1346be2da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.101102 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0eba69-fa4e-471b-8f39-1e1346be2da8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.101376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0eba69-fa4e-471b-8f39-1e1346be2da8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.101640 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0eba69-fa4e-471b-8f39-1e1346be2da8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.104519 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97fc6e-dbe0-445c-a0ab-b4114985b912 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.112346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0eba69-fa4e-471b-8f39-1e1346be2da8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-6b0eba69-fa4e-471b-8f39-1e1346be2da8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:35:45.117091 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:45.118078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 08:35:45.119003 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:45.123076 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 343/683] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Oct 24 08:35:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:35:45.124399 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0eba69-fa4e-471b-8f39-1e1346be2da8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.140456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac08eb6-f73d-4e6b-b1d4-7361938b87bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.141089 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac08eb6-f73d-4e6b-b1d4-7361938b87bd None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.141369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac08eb6-f73d-4e6b-b1d4-7361938b87bd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.141642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac08eb6-f73d-4e6b-b1d4-7361938b87bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.142317 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac08eb6-f73d-4e6b-b1d4-7361938b87bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:45.143749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac08eb6-f73d-4e6b-b1d4-7361938b87bd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:45.170979 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 341/684] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:35:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:35:45.183798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:45.227491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac08eb6-f73d-4e6b-b1d4-7361938b87bd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.247453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0eba69-fa4e-471b-8f39-1e1346be2da8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.298096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a1289-5298-46f9-979c-f514231842be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.299169 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a1289-5298-46f9-979c-f514231842be None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.299751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a1289-5298-46f9-979c-f514231842be None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.300549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a1289-5298-46f9-979c-f514231842be None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.302183 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a1289-5298-46f9-979c-f514231842be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:45.303054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a1289-5298-46f9-979c-f514231842be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:45.305045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a1289-5298-46f9-979c-f514231842be None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:45.321052 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 344/685] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:45.323719 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a1289-5298-46f9-979c-f514231842be None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.363506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac08eb6-f73d-4e6b-b1d4-7361938b87bd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.410201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402497ea-fdf8-40be-8287-5e284078809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.410884 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402497ea-fdf8-40be-8287-5e284078809e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.411158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402497ea-fdf8-40be-8287-5e284078809e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.411798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402497ea-fdf8-40be-8287-5e284078809e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.412377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402497ea-fdf8-40be-8287-5e284078809e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:45.414153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402497ea-fdf8-40be-8287-5e284078809e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-402497ea-fdf8-40be-8287-5e284078809e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:45.424962 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:45.425972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:45.426907 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 342/686] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:35:45.432067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402497ea-fdf8-40be-8287-5e284078809e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.445878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:45.448571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a1289-5298-46f9-979c-f514231842be None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.451267 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:45.512154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85ec4f-afc3-4afd-91c3-675c88e1af2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.512795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85ec4f-afc3-4afd-91c3-675c88e1af2e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.513279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85ec4f-afc3-4afd-91c3-675c88e1af2e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.515967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85ec4f-afc3-4afd-91c3-675c88e1af2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.529622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85ec4f-afc3-4afd-91c3-675c88e1af2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:45.531094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85ec4f-afc3-4afd-91c3-675c88e1af2e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-6d85ec4f-afc3-4afd-91c3-675c88e1af2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:45.546146 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:45.547183 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:45.548506 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:45.548506 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:45.548506 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:45.548506 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 345/687] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:45.553197 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85ec4f-afc3-4afd-91c3-675c88e1af2e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.561232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402497ea-fdf8-40be-8287-5e284078809e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.598385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38303f69-fa76-4ee7-a821-e20135fce9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.599572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38303f69-fa76-4ee7-a821-e20135fce9f7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.600173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38303f69-fa76-4ee7-a821-e20135fce9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.600661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38303f69-fa76-4ee7-a821-e20135fce9f7 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.601478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38303f69-fa76-4ee7-a821-e20135fce9f7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:45.601974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38303f69-fa76-4ee7-a821-e20135fce9f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:45.603758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38303f69-fa76-4ee7-a821-e20135fce9f7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:45.613883 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 343/688] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:45.615551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38303f69-fa76-4ee7-a821-e20135fce9f7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.654796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85ec4f-afc3-4afd-91c3-675c88e1af2e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.689749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246e8c5-6d7b-4b7c-9dff-f4507f814e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.690352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246e8c5-6d7b-4b7c-9dff-f4507f814e8a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.690629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246e8c5-6d7b-4b7c-9dff-f4507f814e8a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.690931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246e8c5-6d7b-4b7c-9dff-f4507f814e8a None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.691684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246e8c5-6d7b-4b7c-9dff-f4507f814e8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:45.691984 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246e8c5-6d7b-4b7c-9dff-f4507f814e8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:45.693521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246e8c5-6d7b-4b7c-9dff-f4507f814e8a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:45.698186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:45.712826 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 346/689] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:45.714009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246e8c5-6d7b-4b7c-9dff-f4507f814e8a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.725742 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38303f69-fa76-4ee7-a821-e20135fce9f7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.733173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:45.764279 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b622eee-e740-4e4d-91c3-4a330840f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.765372 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622eee-e740-4e4d-91c3-4a330840f8cf None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.765943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622eee-e740-4e4d-91c3-4a330840f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.766418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622eee-e740-4e4d-91c3-4a330840f8cf None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.767284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b622eee-e740-4e4d-91c3-4a330840f8cf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:45.767783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b622eee-e740-4e4d-91c3-4a330840f8cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:45.769642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b622eee-e740-4e4d-91c3-4a330840f8cf None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:45.780164 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 344/690] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:45.781724 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b622eee-e740-4e4d-91c3-4a330840f8cf None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.821228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246e8c5-6d7b-4b7c-9dff-f4507f814e8a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.857116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf88d3-6d44-445d-aac3-ee9fe07e49ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.857668 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf88d3-6d44-445d-aac3-ee9fe07e49ad None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.857943 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf88d3-6d44-445d-aac3-ee9fe07e49ad None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.858211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf88d3-6d44-445d-aac3-ee9fe07e49ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.865799 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf88d3-6d44-445d-aac3-ee9fe07e49ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:45.867438 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf88d3-6d44-445d-aac3-ee9fe07e49ad None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-eedf88d3-6d44-445d-aac3-ee9fe07e49ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:45.879563 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:45.880518 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 08:35:45.881475 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:45.885550 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 347/691] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:45.886591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf88d3-6d44-445d-aac3-ee9fe07e49ad None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.890241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b622eee-e740-4e4d-91c3-4a330840f8cf None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:45.927232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d0269-de4f-4cc6-b251-3595aa0e90ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:45.928215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d0269-de4f-4cc6-b251-3595aa0e90ff None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:45.928688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d0269-de4f-4cc6-b251-3595aa0e90ff None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:45.929195 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d0269-de4f-4cc6-b251-3595aa0e90ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:45.930081 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d0269-de4f-4cc6-b251-3595aa0e90ff None admin] RBAC: Adding query filter params (interface=public, service_id=e6dbaf04e5884575827bcf90c0514cfc, region_id=RegionOne) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:45.930601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d0269-de4f-4cc6-b251-3595aa0e90ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:45.932345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d0269-de4f-4cc6-b251-3595aa0e90ff None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:45.942639 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 345/692] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/endpoints?service_id=e6dbaf04e5884575827bcf90c0514cfc&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:45.944481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d0269-de4f-4cc6-b251-3595aa0e90ff None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:45.947494 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:45.969627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:45.994755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf88d3-6d44-445d-aac3-ee9fe07e49ad None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:46.030865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643d3df-5329-4f2d-a422-cfbf4dde2532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:46.031651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643d3df-5329-4f2d-a422-cfbf4dde2532 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:46.031829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643d3df-5329-4f2d-a422-cfbf4dde2532 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:46.032096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643d3df-5329-4f2d-a422-cfbf4dde2532 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:46.032818 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643d3df-5329-4f2d-a422-cfbf4dde2532 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:46.033175 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643d3df-5329-4f2d-a422-cfbf4dde2532 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:46.034586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643d3df-5329-4f2d-a422-cfbf4dde2532 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:46.055564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d0269-de4f-4cc6-b251-3595aa0e90ff None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:46.059019 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 348/693] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:46.059778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643d3df-5329-4f2d-a422-cfbf4dde2532 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:46.092345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8db302-43b9-4ce1-b0bd-830d46d040d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:46.093316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db302-43b9-4ce1-b0bd-830d46d040d1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:46.093809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db302-43b9-4ce1-b0bd-830d46d040d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:46.094465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db302-43b9-4ce1-b0bd-830d46d040d1 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:46.095775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8db302-43b9-4ce1-b0bd-830d46d040d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:46.097357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8db302-43b9-4ce1-b0bd-830d46d040d1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:46.111093 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 346/694] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Oct 24 08:35:45 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:35:46.168620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643d3df-5329-4f2d-a422-cfbf4dde2532 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:46.208930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad88ee-f2d4-40da-ab98-4e524bae71ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:46.209557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad88ee-f2d4-40da-ab98-4e524bae71ae None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:46.209830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad88ee-f2d4-40da-ab98-4e524bae71ae None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:46.210164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad88ee-f2d4-40da-ab98-4e524bae71ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:46.213195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:46.290755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad88ee-f2d4-40da-ab98-4e524bae71ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:46.290755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad88ee-f2d4-40da-ab98-4e524bae71ae None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:46.386029 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 349/695] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:35:46 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:35:46.391708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8db302-43b9-4ce1-b0bd-830d46d040d1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:46.460596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:46.500847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8db302-43b9-4ce1-b0bd-830d46d040d1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:46.535179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8100f-e804-4256-aee5-c5af326d5f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:46.536120 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8100f-e804-4256-aee5-c5af326d5f4f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:46.536927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8100f-e804-4256-aee5-c5af326d5f4f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:46.537367 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8100f-e804-4256-aee5-c5af326d5f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:46.548122 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8100f-e804-4256-aee5-c5af326d5f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:46.549824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8100f-e804-4256-aee5-c5af326d5f4f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-78a8100f-e804-4256-aee5-c5af326d5f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:35:46.569025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:46.570033 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:35:46.570972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:35:46.570972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:35:46.570972 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:46.570972 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 347/696] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:46.577699 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad88ee-f2d4-40da-ab98-4e524bae71ae None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:46.590395 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c928a9-7d7e-4970-8740-0648aeeac17e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:46.590695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c928a9-7d7e-4970-8740-0648aeeac17e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:46.591001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c928a9-7d7e-4970-8740-0648aeeac17e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:46.592481 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 348/697] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:35:46.597844 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558c6b1-c11c-4775-86e2-48b204b8abb7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:46.598359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558c6b1-c11c-4775-86e2-48b204b8abb7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:46.598881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558c6b1-c11c-4775-86e2-48b204b8abb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:46.612966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:46.619194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8558c6b1-c11c-4775-86e2-48b204b8abb7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:46.620735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8558c6b1-c11c-4775-86e2-48b204b8abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:46.688687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad88ee-f2d4-40da-ab98-4e524bae71ae None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:46.711737 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 349/698] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:46.742261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee68dd-044f-4289-a684-487ba0621022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:46.743331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee68dd-044f-4289-a684-487ba0621022 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:46.743836 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee68dd-044f-4289-a684-487ba0621022 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:46.744350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee68dd-044f-4289-a684-487ba0621022 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:46.745292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee68dd-044f-4289-a684-487ba0621022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:46.745802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee68dd-044f-4289-a684-487ba0621022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:46.749341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee68dd-044f-4289-a684-487ba0621022 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:46.762275 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 350/699] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:46.769928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558c6b1-c11c-4775-86e2-48b204b8abb7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:46.919119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:47.027714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee68dd-044f-4289-a684-487ba0621022 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:47.028973 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558c6b1-c11c-4775-86e2-48b204b8abb7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:47.084713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07916d-fe90-4488-a6f7-8a45780c8120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:47.085351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07916d-fe90-4488-a6f7-8a45780c8120 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.085624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07916d-fe90-4488-a6f7-8a45780c8120 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.085979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07916d-fe90-4488-a6f7-8a45780c8120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.096318 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07916d-fe90-4488-a6f7-8a45780c8120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:47.097936 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07916d-fe90-4488-a6f7-8a45780c8120 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ad07916d-fe90-4488-a6f7-8a45780c8120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:47.114749 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:47.115705 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:47.116610 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:47.116610 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:47.116610 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:47.116610 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:47.116610 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:47.116610 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:47.116610 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:47.116610 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:47.116610 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 350/700] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:47.123196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07916d-fe90-4488-a6f7-8a45780c8120 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:47.204880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:47.211619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee68dd-044f-4289-a684-487ba0621022 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:47.248037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955753c-ce89-4c51-8487-684b9cf9451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:47.249052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955753c-ce89-4c51-8487-684b9cf9451e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.249497 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955753c-ce89-4c51-8487-684b9cf9451e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.249947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955753c-ce89-4c51-8487-684b9cf9451e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.250906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955753c-ce89-4c51-8487-684b9cf9451e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:47.252459 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955753c-ce89-4c51-8487-684b9cf9451e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:47.255470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07916d-fe90-4488-a6f7-8a45780c8120 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c955753c-ce89-4c51-8487-684b9cf9451e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:47.263445 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:47.264654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:47.265618 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:47.278851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:47.279030 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 351/701] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:35:47.279030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955753c-ce89-4c51-8487-684b9cf9451e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:47.314665 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ae622-ee74-497f-8a5d-314beac0582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:47.315606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ae622-ee74-497f-8a5d-314beac0582f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.316577 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ae622-ee74-497f-8a5d-314beac0582f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.317022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ae622-ee74-497f-8a5d-314beac0582f None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.317880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ae622-ee74-497f-8a5d-314beac0582f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:47.318763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ae622-ee74-497f-8a5d-314beac0582f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:47.321022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ae622-ee74-497f-8a5d-314beac0582f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:47.337146 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 351/702] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:35:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:47.342577 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ae622-ee74-497f-8a5d-314beac0582f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:47.394697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955753c-ce89-4c51-8487-684b9cf9451e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:47.437639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83546116-c330-406f-ad83-8d904c4e73cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:47.438678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83546116-c330-406f-ad83-8d904c4e73cf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.439188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83546116-c330-406f-ad83-8d904c4e73cf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.439625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83546116-c330-406f-ad83-8d904c4e73cf None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.440412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83546116-c330-406f-ad83-8d904c4e73cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:47.440974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83546116-c330-406f-ad83-8d904c4e73cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:47.442495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83546116-c330-406f-ad83-8d904c4e73cf None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:47.453092 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 352/703] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:47.454702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83546116-c330-406f-ad83-8d904c4e73cf None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:47.458019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ae622-ee74-497f-8a5d-314beac0582f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:47.482895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:47.493879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb22be-2928-4a6f-bc8e-d102d0d950c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:47.494486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb22be-2928-4a6f-bc8e-d102d0d950c4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.494787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb22be-2928-4a6f-bc8e-d102d0d950c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.495096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb22be-2928-4a6f-bc8e-d102d0d950c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.495823 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb22be-2928-4a6f-bc8e-d102d0d950c4 None admin] RBAC: Adding query filter params (role.id=1c0b192e409a435a98f939e77554596d, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=771fc0da7a8749f0b80ce0630951500a) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:47.496126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb22be-2928-4a6f-bc8e-d102d0d950c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:47.497635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb22be-2928-4a6f-bc8e-d102d0d950c4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:47.504462 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 352/704] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:35:47 2022] GET /identity/v3/role_assignments?user.id=771fc0da7a8749f0b80ce0630951500a&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=1c0b192e409a435a98f939e77554596d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:47.523781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3f28a-a35a-4712-8988-af0399bbcf96 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.524115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3f28a-a35a-4712-8988-af0399bbcf96 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.524334 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3f28a-a35a-4712-8988-af0399bbcf96 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.526022 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 353/705] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:35:47.532820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ffcdc-06f0-4fe6-a1fd-acd84216ed93 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.533173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ffcdc-06f0-4fe6-a1fd-acd84216ed93 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.533569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ffcdc-06f0-4fe6-a1fd-acd84216ed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.553410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-525ffcdc-06f0-4fe6-a1fd-acd84216ed93 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:47.554794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-525ffcdc-06f0-4fe6-a1fd-acd84216ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:47.585565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:47.598301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83546116-c330-406f-ad83-8d904c4e73cf None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:47.639377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc315-4c9c-4ad1-a983-dd2f78cb6d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:47.640329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc315-4c9c-4ad1-a983-dd2f78cb6d7a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.640739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc315-4c9c-4ad1-a983-dd2f78cb6d7a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.641175 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc315-4c9c-4ad1-a983-dd2f78cb6d7a None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.641958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dc315-4c9c-4ad1-a983-dd2f78cb6d7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:47.642390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dc315-4c9c-4ad1-a983-dd2f78cb6d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:47.643340 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 354/706] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:47.644210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dc315-4c9c-4ad1-a983-dd2f78cb6d7a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:47.650691 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 353/707] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:47.656596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ffcdc-06f0-4fe6-a1fd-acd84216ed93 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:47.765548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ffcdc-06f0-4fe6-a1fd-acd84216ed93 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:47.800512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988487c-5c94-49b8-a8ae-85108a518f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:47.801051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988487c-5c94-49b8-a8ae-85108a518f19 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.801327 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988487c-5c94-49b8-a8ae-85108a518f19 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.801596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988487c-5c94-49b8-a8ae-85108a518f19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.802293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988487c-5c94-49b8-a8ae-85108a518f19 None admin] RBAC: Adding query filter params (interface=public, service_id=8af331bee91c44fdb18575a4494011d1, region_id=RegionOne) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:47.802597 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988487c-5c94-49b8-a8ae-85108a518f19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:47.804038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988487c-5c94-49b8-a8ae-85108a518f19 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:47.809806 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 355/708] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Oct 24 08:35:47 2022] GET /identity/v3/endpoints?service_id=8af331bee91c44fdb18575a4494011d1&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:47.815513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc315-4c9c-4ad1-a983-dd2f78cb6d7a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:47.909783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988487c-5c94-49b8-a8ae-85108a518f19 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:47.913672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:47.930085 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc315-4c9c-4ad1-a983-dd2f78cb6d7a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:47.941813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:47.971859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906f30d-f437-46c5-95d3-954940a2e64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:47.972535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906f30d-f437-46c5-95d3-954940a2e64f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:47.973106 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906f30d-f437-46c5-95d3-954940a2e64f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:47.974002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906f30d-f437-46c5-95d3-954940a2e64f None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:47.974665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906f30d-f437-46c5-95d3-954940a2e64f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:47.976239 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906f30d-f437-46c5-95d3-954940a2e64f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:47.985525 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 354/709] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Oct 24 08:35:47 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:35:48.032067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988487c-5c94-49b8-a8ae-85108a518f19 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:48.071702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1981e9c-d47c-4018-8c7a-439eed5b830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:48.072328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1981e9c-d47c-4018-8c7a-439eed5b830f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:48.072670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1981e9c-d47c-4018-8c7a-439eed5b830f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:48.072948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1981e9c-d47c-4018-8c7a-439eed5b830f None admin] PATH_INFO: `/v3/services/compute` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:48.073615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1981e9c-d47c-4018-8c7a-439eed5b830f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:48.075061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1981e9c-d47c-4018-8c7a-439eed5b830f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f1981e9c-d47c-4018-8c7a-439eed5b830f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:48.084906 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:48.085990 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:48.086864 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 356/710] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:35:48.092329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906f30d-f437-46c5-95d3-954940a2e64f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:48.203930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906f30d-f437-46c5-95d3-954940a2e64f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:48.238082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57df8b3-eb52-4e1f-97be-df849bf8c517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:48.238691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57df8b3-eb52-4e1f-97be-df849bf8c517 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:48.238988 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57df8b3-eb52-4e1f-97be-df849bf8c517 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:48.239257 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57df8b3-eb52-4e1f-97be-df849bf8c517 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:48.239884 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57df8b3-eb52-4e1f-97be-df849bf8c517 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:48.240182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57df8b3-eb52-4e1f-97be-df849bf8c517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:48.241535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57df8b3-eb52-4e1f-97be-df849bf8c517 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:48.246854 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 355/711] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:48.252484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1981e9c-d47c-4018-8c7a-439eed5b830f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:48.360632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:48.370550 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1981e9c-d47c-4018-8c7a-439eed5b830f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:48.405057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47854dc-de45-4859-bece-86fa48302424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:48.405727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47854dc-de45-4859-bece-86fa48302424 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:48.406010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47854dc-de45-4859-bece-86fa48302424 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:48.406277 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47854dc-de45-4859-bece-86fa48302424 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:48.406932 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47854dc-de45-4859-bece-86fa48302424 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:48.407234 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47854dc-de45-4859-bece-86fa48302424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:48.408711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47854dc-de45-4859-bece-86fa48302424 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:48.414138 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 357/712] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:48.442594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc6d0f-75fb-4171-bd55-a279550ca592 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:48.442901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc6d0f-75fb-4171-bd55-a279550ca592 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:48.443218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc6d0f-75fb-4171-bd55-a279550ca592 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:48.444820 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 356/713] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:48.450438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7f5ec-ed08-4b5c-b0f7-8e2ee51c9c23 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:48.451039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7f5ec-ed08-4b5c-b0f7-8e2ee51c9c23 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:48.451507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7f5ec-ed08-4b5c-b0f7-8e2ee51c9c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:48.471914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fce7f5ec-ed08-4b5c-b0f7-8e2ee51c9c23 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:48.473437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fce7f5ec-ed08-4b5c-b0f7-8e2ee51c9c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:48.549883 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 358/714] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:48.813805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc6d0f-75fb-4171-bd55-a279550ca592 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:48.830472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:48.916302 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc6d0f-75fb-4171-bd55-a279550ca592 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:48.954080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d78c3-9150-49f1-bb54-afcaecc0be9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:48.954749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d78c3-9150-49f1-bb54-afcaecc0be9f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:48.955031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d78c3-9150-49f1-bb54-afcaecc0be9f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:48.955302 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d78c3-9150-49f1-bb54-afcaecc0be9f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:48.955890 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d78c3-9150-49f1-bb54-afcaecc0be9f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:48.957292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d78c3-9150-49f1-bb54-afcaecc0be9f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d52d78c3-9150-49f1-bb54-afcaecc0be9f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:48.966729 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:48.967726 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:48.968581 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 357/715] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:48.974032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7f5ec-ed08-4b5c-b0f7-8e2ee51c9c23 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:49.056754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:49.072927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7f5ec-ed08-4b5c-b0f7-8e2ee51c9c23 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:49.105815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aeb6b3-d0b9-4c09-a864-942e8fe16299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:49.106430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aeb6b3-d0b9-4c09-a864-942e8fe16299 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:49.106707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aeb6b3-d0b9-4c09-a864-942e8fe16299 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:49.107001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aeb6b3-d0b9-4c09-a864-942e8fe16299 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:49.107625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aeb6b3-d0b9-4c09-a864-942e8fe16299 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:49.107924 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aeb6b3-d0b9-4c09-a864-942e8fe16299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:49.109306 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aeb6b3-d0b9-4c09-a864-942e8fe16299 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:49.114329 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 359/716] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:49.120072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d78c3-9150-49f1-bb54-afcaecc0be9f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:49.198193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:49.220394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d78c3-9150-49f1-bb54-afcaecc0be9f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:49.256783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fbbbb-7363-47bf-8cef-53e92ba943b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:49.257384 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fbbbb-7363-47bf-8cef-53e92ba943b7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:49.257660 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fbbbb-7363-47bf-8cef-53e92ba943b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:49.257928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fbbbb-7363-47bf-8cef-53e92ba943b7 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:49.258553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fbbbb-7363-47bf-8cef-53e92ba943b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:49.258880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fbbbb-7363-47bf-8cef-53e92ba943b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:49.260305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fbbbb-7363-47bf-8cef-53e92ba943b7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:49.265590 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 358/717] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Oct 24 08:35:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:49.271411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aeb6b3-d0b9-4c09-a864-942e8fe16299 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:49.387419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aeb6b3-d0b9-4c09-a864-942e8fe16299 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:49.424979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488674bb-1fb8-4ae8-a81f-128d1346ecf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:49.425600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488674bb-1fb8-4ae8-a81f-128d1346ecf1 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:49.425876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488674bb-1fb8-4ae8-a81f-128d1346ecf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:49.426144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488674bb-1fb8-4ae8-a81f-128d1346ecf1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:49.426848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488674bb-1fb8-4ae8-a81f-128d1346ecf1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:49.428215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488674bb-1fb8-4ae8-a81f-128d1346ecf1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:49.449686 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 360/718] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:35:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:49.832233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74759809-58de-45c3-8e4e-583e34306c32 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:49.832898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74759809-58de-45c3-8e4e-583e34306c32 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:49.833391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74759809-58de-45c3-8e4e-583e34306c32 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:49.835198 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 359/719] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:49.840753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2385d9-c542-4593-a38c-e35a21728ece None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:49.841269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2385d9-c542-4593-a38c-e35a21728ece None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:49.841739 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2385d9-c542-4593-a38c-e35a21728ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:49.849960 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:49.860679 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fc2385d9-c542-4593-a38c-e35a21728ece None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:49.862708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fc2385d9-c542-4593-a38c-e35a21728ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:49.937511 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 361/720] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:50.216088 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74759809-58de-45c3-8e4e-583e34306c32 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:50.297305 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477941f-8546-4daf-a5ac-28b8f200b2c5 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:50.297817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477941f-8546-4daf-a5ac-28b8f200b2c5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:50.297895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477941f-8546-4daf-a5ac-28b8f200b2c5 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:50.299541 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 362/721] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:50.304833 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc91651-29b5-4284-afae-07bd01b0fa89 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:50.305110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc91651-29b5-4284-afae-07bd01b0fa89 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:50.305390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc91651-29b5-4284-afae-07bd01b0fa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:50.324243 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bfc91651-29b5-4284-afae-07bd01b0fa89 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:50.325426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bfc91651-29b5-4284-afae-07bd01b0fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:50.330507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74759809-58de-45c3-8e4e-583e34306c32 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:50.365835 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0046423-5a8c-4d6e-8502-5859f0394043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:50.366474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0046423-5a8c-4d6e-8502-5859f0394043 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:50.366770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0046423-5a8c-4d6e-8502-5859f0394043 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:50.367043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0046423-5a8c-4d6e-8502-5859f0394043 None admin] PATH_INFO: `/v3/services/image` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:50.367631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0046423-5a8c-4d6e-8502-5859f0394043 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:50.369163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0046423-5a8c-4d6e-8502-5859f0394043 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f0046423-5a8c-4d6e-8502-5859f0394043 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:50.379229 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:50.380340 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:50.381211 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 360/722] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Oct 24 08:35:50 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:50.386615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0046423-5a8c-4d6e-8502-5859f0394043 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:50.396530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:50.410843 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 363/723] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:50.485859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0046423-5a8c-4d6e-8502-5859f0394043 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:50.519146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777925a-846c-4c89-b313-80659d129a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:50.519746 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777925a-846c-4c89-b313-80659d129a65 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:50.520022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777925a-846c-4c89-b313-80659d129a65 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:50.520291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777925a-846c-4c89-b313-80659d129a65 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:50.520909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777925a-846c-4c89-b313-80659d129a65 None admin] RBAC: Adding query filter params (name=image) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:50.521208 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777925a-846c-4c89-b313-80659d129a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:50.522564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777925a-846c-4c89-b313-80659d129a65 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:50.527614 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 361/724] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:50.533302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc91651-29b5-4284-afae-07bd01b0fa89 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:50.595584 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:50.639811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc91651-29b5-4284-afae-07bd01b0fa89 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:50.678562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014f64e-bf8a-44af-9621-bede4fd1da16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:50.679211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014f64e-bf8a-44af-9621-bede4fd1da16 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:50.679488 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014f64e-bf8a-44af-9621-bede4fd1da16 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:50.679770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014f64e-bf8a-44af-9621-bede4fd1da16 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:50.680388 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014f64e-bf8a-44af-9621-bede4fd1da16 None admin] RBAC: Adding query filter params (type=image) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:50.680688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014f64e-bf8a-44af-9621-bede4fd1da16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:50.682066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014f64e-bf8a-44af-9621-bede4fd1da16 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:50.687218 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 364/725] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:50.691150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777925a-846c-4c89-b313-80659d129a65 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:50.718805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0671df-28ec-481f-a4e0-27bdd56932b0 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:50.719337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0671df-28ec-481f-a4e0-27bdd56932b0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:50.719783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0671df-28ec-481f-a4e0-27bdd56932b0 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:50.721653 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 365/726] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:50.727639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a034d07-86b5-487f-87eb-1593751b3215 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:50.728166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a034d07-86b5-487f-87eb-1593751b3215 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:50.728585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a034d07-86b5-487f-87eb-1593751b3215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:50.748598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5a034d07-86b5-487f-87eb-1593751b3215 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:50.750069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5a034d07-86b5-487f-87eb-1593751b3215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:50.773242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:50.808882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777925a-846c-4c89-b313-80659d129a65 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:50.831159 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 366/727] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:50.847436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd173745-9f72-4de7-be0c-0be6cd637560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:50.848791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd173745-9f72-4de7-be0c-0be6cd637560 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:50.849402 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd173745-9f72-4de7-be0c-0be6cd637560 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:50.849993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd173745-9f72-4de7-be0c-0be6cd637560 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:50.851032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd173745-9f72-4de7-be0c-0be6cd637560 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:50.852662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd173745-9f72-4de7-be0c-0be6cd637560 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-fd173745-9f72-4de7-be0c-0be6cd637560 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:50.862441 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:50.863581 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:50.864425 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:50.867209 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 362/728] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:50.873192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a034d07-86b5-487f-87eb-1593751b3215 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:50.935076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:50.976578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a034d07-86b5-487f-87eb-1593751b3215 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:51.011531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa36a0-4b65-4b12-8037-296db3b7683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:51.012072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa36a0-4b65-4b12-8037-296db3b7683c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:51.012351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa36a0-4b65-4b12-8037-296db3b7683c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:51.012623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa36a0-4b65-4b12-8037-296db3b7683c None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:51.013251 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa36a0-4b65-4b12-8037-296db3b7683c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:51.013554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa36a0-4b65-4b12-8037-296db3b7683c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:51.014943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa36a0-4b65-4b12-8037-296db3b7683c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:51.020104 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 367/729] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:51.025837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd173745-9f72-4de7-be0c-0be6cd637560 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:51.107697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa36a0-4b65-4b12-8037-296db3b7683c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:51.133247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd173745-9f72-4de7-be0c-0be6cd637560 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:51.170605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cec270-211d-4e3f-9ea2-c3dadb2bffc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:51.171241 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cec270-211d-4e3f-9ea2-c3dadb2bffc3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:51.171515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cec270-211d-4e3f-9ea2-c3dadb2bffc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:51.171780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cec270-211d-4e3f-9ea2-c3dadb2bffc3 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:51.172406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cec270-211d-4e3f-9ea2-c3dadb2bffc3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:51.172703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cec270-211d-4e3f-9ea2-c3dadb2bffc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:51.174207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cec270-211d-4e3f-9ea2-c3dadb2bffc3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:51.175844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:51.181426 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 363/730] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:51.186923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cec270-211d-4e3f-9ea2-c3dadb2bffc3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:51.221206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa36a0-4b65-4b12-8037-296db3b7683c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:51.248525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:51.264822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5667bb-a9e6-4fcf-8bc4-dfb0d9d267bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:51.264822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5667bb-a9e6-4fcf-8bc4-dfb0d9d267bf None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:51.264822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5667bb-a9e6-4fcf-8bc4-dfb0d9d267bf None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:51.264822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5667bb-a9e6-4fcf-8bc4-dfb0d9d267bf None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:51.265296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5667bb-a9e6-4fcf-8bc4-dfb0d9d267bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:51.266801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5667bb-a9e6-4fcf-8bc4-dfb0d9d267bf None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:51.290745 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 368/731] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:35:51 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:35:51.306656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cec270-211d-4e3f-9ea2-c3dadb2bffc3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:51.352782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab684b1-b95f-4d59-a06e-697d0c28bc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:51.353507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab684b1-b95f-4d59-a06e-697d0c28bc1e None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:51.353787 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab684b1-b95f-4d59-a06e-697d0c28bc1e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:51.354055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab684b1-b95f-4d59-a06e-697d0c28bc1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:51.354825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab684b1-b95f-4d59-a06e-697d0c28bc1e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:51.356307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab684b1-b95f-4d59-a06e-697d0c28bc1e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:51.383283 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 364/732] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:35:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:51.741868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960292a-3062-4e5a-bb7b-bf51a8e38268 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:51.742427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960292a-3062-4e5a-bb7b-bf51a8e38268 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:51.742427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960292a-3062-4e5a-bb7b-bf51a8e38268 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:51.743928 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 369/733] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:51.749514 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6a631-cd51-4a5e-b47b-83b7fdbe1d99 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:51.750040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6a631-cd51-4a5e-b47b-83b7fdbe1d99 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:51.750517 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6a631-cd51-4a5e-b47b-83b7fdbe1d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:51.769304 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-dbc6a631-cd51-4a5e-b47b-83b7fdbe1d99 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:51.770813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dbc6a631-cd51-4a5e-b47b-83b7fdbe1d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:51.819343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:51.849624 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 365/734] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:52.119393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960292a-3062-4e5a-bb7b-bf51a8e38268 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:52.223239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960292a-3062-4e5a-bb7b-bf51a8e38268 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:52.259459 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:52.260083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:52.260439 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:52.260725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:52.261385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:52.262837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:52.265450 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING keystone.server.flask.common [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 24 08:35:52.291188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.sql.core [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 08:35:52.291188 dv-cinder-master-3-smb devstack@keystone.service[87873]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 08:35:52.291188 dv-cinder-master-3-smb devstack@keystone.service[87873]: [parameters: {'id': '79dd95aaea1d4cdb9a0180f84c0fddd5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 24 08:35:52.291188 dv-cinder-master-3-smb devstack@keystone.service[87873]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87873) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 79dd95aaea1d4cdb9a0180f84c0fddd5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 79dd95aaea1d4cdb9a0180f84c0fddd5. Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:35:52.298358 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 08:35:52.299328 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application yield resource Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise self.value Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application session.commit() Oct 24 08:35:52.300350 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.flush() Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application compat.raise_( Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 08:35:52.301280 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 08:35:52.302209 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:35:52.304078 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [parameters: {'id': '79dd95aaea1d4cdb9a0180f84c0fddd5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:52.308520 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:52.310559 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 79dd95aaea1d4cdb9a0180f84c0fddd5. Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:52.312823 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 370/735] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Oct 24 08:35:52 2022] POST /identity/v3/projects => generated 262 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:52.318786 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6a631-cd51-4a5e-b47b-83b7fdbe1d99 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:52.424366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6a631-cd51-4a5e-b47b-83b7fdbe1d99 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:52.458339 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802807d-cd95-44eb-a48a-245e711d195f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:52.459002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802807d-cd95-44eb-a48a-245e711d195f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:52.459284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802807d-cd95-44eb-a48a-245e711d195f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:52.459561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802807d-cd95-44eb-a48a-245e711d195f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:52.466365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802807d-cd95-44eb-a48a-245e711d195f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:52.467852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802807d-cd95-44eb-a48a-245e711d195f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c802807d-cd95-44eb-a48a-245e711d195f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:35:52.478939 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:52.479895 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:52.480797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:35:52.480797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:35:52.480797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:35:52.480797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:35:52.480797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:35:52.480797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:35:52.480797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:35:52.480797 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:52.480797 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 366/736] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:52.487917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:52.523222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:52.595078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ac6c3-9c4b-4944-a160-b0d7508bc8ef None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:52.630118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0060f-6839-4381-9fb2-e80e3c590fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:52.631245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0060f-6839-4381-9fb2-e80e3c590fb7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:52.631718 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0060f-6839-4381-9fb2-e80e3c590fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:52.632204 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0060f-6839-4381-9fb2-e80e3c590fb7 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:52.633053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0060f-6839-4381-9fb2-e80e3c590fb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:52.633562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0060f-6839-4381-9fb2-e80e3c590fb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:52.635449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0060f-6839-4381-9fb2-e80e3c590fb7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:52.656592 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 371/737] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:52.912532 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d455c6d-dadb-407a-9d56-2bd03aa1f9c4 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:52.912908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d455c6d-dadb-407a-9d56-2bd03aa1f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:52.913081 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d455c6d-dadb-407a-9d56-2bd03aa1f9c4 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:52.914667 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 367/738] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:52.920475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1691c3-10bf-459e-98ef-5df81460fc67 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:52.920924 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1691c3-10bf-459e-98ef-5df81460fc67 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:52.921099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1691c3-10bf-459e-98ef-5df81460fc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:52.940804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-de1691c3-10bf-459e-98ef-5df81460fc67 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:52.942297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-de1691c3-10bf-459e-98ef-5df81460fc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:53.014832 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 372/739] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:35:53.278278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d455c6d-dadb-407a-9d56-2bd03aa1f9c4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:53.316045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:53.382849 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d455c6d-dadb-407a-9d56-2bd03aa1f9c4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:53.421697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382117cb-1855-4dcb-b5a1-558c8c011478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:53.422341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382117cb-1855-4dcb-b5a1-558c8c011478 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:53.422625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382117cb-1855-4dcb-b5a1-558c8c011478 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:53.422926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382117cb-1855-4dcb-b5a1-558c8c011478 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:53.423616 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382117cb-1855-4dcb-b5a1-558c8c011478 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:53.425054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382117cb-1855-4dcb-b5a1-558c8c011478 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:53.444950 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 368/740] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:35:53 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:53.540698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fec1e6-15c2-478c-92c2-6b8e2713a867 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:53.540890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fec1e6-15c2-478c-92c2-6b8e2713a867 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:53.541251 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fec1e6-15c2-478c-92c2-6b8e2713a867 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:53.542782 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 373/741] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:53.548722 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aec657-f00a-4dc8-9bd0-b2a77f92bcd0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:53.549011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aec657-f00a-4dc8-9bd0-b2a77f92bcd0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:53.549286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aec657-f00a-4dc8-9bd0-b2a77f92bcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:53.570270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-72aec657-f00a-4dc8-9bd0-b2a77f92bcd0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:53.571626 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-72aec657-f00a-4dc8-9bd0-b2a77f92bcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:53.659401 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 369/742] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:53 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:53.714363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c83d65-6d33-41e1-a65c-70f174993333 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:53.714698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c83d65-6d33-41e1-a65c-70f174993333 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:53.715095 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c83d65-6d33-41e1-a65c-70f174993333 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:53.716584 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 374/743] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:53.722101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887b583-6159-4ffb-8c03-d049bcec1c96 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:53.722387 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887b583-6159-4ffb-8c03-d049bcec1c96 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:53.722658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887b583-6159-4ffb-8c03-d049bcec1c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:53.741244 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7887b583-6159-4ffb-8c03-d049bcec1c96 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:53.742785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7887b583-6159-4ffb-8c03-d049bcec1c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:53.760004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:53.811795 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 370/744] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:53 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:53.931387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c83d65-6d33-41e1-a65c-70f174993333 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:53.957223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:54.041761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c83d65-6d33-41e1-a65c-70f174993333 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.076561 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb23d7-bbd2-4aff-b435-0302f4269fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.077117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb23d7-bbd2-4aff-b435-0302f4269fe3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.077392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb23d7-bbd2-4aff-b435-0302f4269fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.077662 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb23d7-bbd2-4aff-b435-0302f4269fe3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.078240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb23d7-bbd2-4aff-b435-0302f4269fe3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.079700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb23d7-bbd2-4aff-b435-0302f4269fe3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.079905 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887b583-6159-4ffb-8c03-d049bcec1c96 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-8afb23d7-bbd2-4aff-b435-0302f4269fe3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:54.089166 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:54.090194 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:54.091063 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 375/745] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:54.096486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb23d7-bbd2-4aff-b435-0302f4269fe3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:54.190228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:54.194648 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887b583-6159-4ffb-8c03-d049bcec1c96 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.205079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb23d7-bbd2-4aff-b435-0302f4269fe3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.236709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdfab4-4b75-426c-92a4-0be248c5415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.237372 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdfab4-4b75-426c-92a4-0be248c5415e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.237612 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdfab4-4b75-426c-92a4-0be248c5415e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.237883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdfab4-4b75-426c-92a4-0be248c5415e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.238463 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdfab4-4b75-426c-92a4-0be248c5415e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.239870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdfab4-4b75-426c-92a4-0be248c5415e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.241853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b50c25-2a23-4c9a-84af-eb6d6b1d4ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.242790 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b50c25-2a23-4c9a-84af-eb6d6b1d4ec7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.243256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b50c25-2a23-4c9a-84af-eb6d6b1d4ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.243707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b50c25-2a23-4c9a-84af-eb6d6b1d4ec7 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.244526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b50c25-2a23-4c9a-84af-eb6d6b1d4ec7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:54.245054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b50c25-2a23-4c9a-84af-eb6d6b1d4ec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.246598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b50c25-2a23-4c9a-84af-eb6d6b1d4ec7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-06cdfab4-4b75-426c-92a4-0be248c5415e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:54.250022 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:54.251042 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:54.251881 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 371/746] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:54.255426 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 376/747] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:54.265946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdfab4-4b75-426c-92a4-0be248c5415e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:54.267109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b50c25-2a23-4c9a-84af-eb6d6b1d4ec7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:54.279524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:54.369358 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdfab4-4b75-426c-92a4-0be248c5415e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.392493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b50c25-2a23-4c9a-84af-eb6d6b1d4ec7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.412102 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c733c-d0e8-41cc-97fe-9d8d124f32e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.413327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c733c-d0e8-41cc-97fe-9d8d124f32e8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.414177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c733c-d0e8-41cc-97fe-9d8d124f32e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.414597 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c733c-d0e8-41cc-97fe-9d8d124f32e8 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.415656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c733c-d0e8-41cc-97fe-9d8d124f32e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:54.416100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c733c-d0e8-41cc-97fe-9d8d124f32e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.418258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c733c-d0e8-41cc-97fe-9d8d124f32e8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.424877 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 372/748] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:54.430777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c733c-d0e8-41cc-97fe-9d8d124f32e8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:54.434158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ca297-22e8-4a76-8077-51606acb3553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.435308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ca297-22e8-4a76-8077-51606acb3553 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.435308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ca297-22e8-4a76-8077-51606acb3553 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.435643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ca297-22e8-4a76-8077-51606acb3553 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.436375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ca297-22e8-4a76-8077-51606acb3553 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:54.436780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ca297-22e8-4a76-8077-51606acb3553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.438248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ca297-22e8-4a76-8077-51606acb3553 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.448669 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 377/749] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:54.449895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ca297-22e8-4a76-8077-51606acb3553 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:54.473250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:54.525492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:54.549194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c733c-d0e8-41cc-97fe-9d8d124f32e8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.571999 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ca297-22e8-4a76-8077-51606acb3553 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.589290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381e65c-0acf-4b7d-b054-f07af04092c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.590493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381e65c-0acf-4b7d-b054-f07af04092c8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.591001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381e65c-0acf-4b7d-b054-f07af04092c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.591468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381e65c-0acf-4b7d-b054-f07af04092c8 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.592285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381e65c-0acf-4b7d-b054-f07af04092c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:54.593216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381e65c-0acf-4b7d-b054-f07af04092c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.595090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381e65c-0acf-4b7d-b054-f07af04092c8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.601353 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 373/750] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:54.607505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381e65c-0acf-4b7d-b054-f07af04092c8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:54.611947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913506a8-6442-4934-85f1-a838970f4abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.612198 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913506a8-6442-4934-85f1-a838970f4abb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.612449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913506a8-6442-4934-85f1-a838970f4abb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.612724 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913506a8-6442-4934-85f1-a838970f4abb None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.613426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913506a8-6442-4934-85f1-a838970f4abb None admin] RBAC: Adding query filter params (interface=public, service_id=54dee9ae5c804df38a4e6032251a6cfc, region_id=RegionOne) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:54.613735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913506a8-6442-4934-85f1-a838970f4abb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.615148 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913506a8-6442-4934-85f1-a838970f4abb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.625508 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 378/751] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Oct 24 08:35:54 2022] GET /identity/v3/endpoints?service_id=54dee9ae5c804df38a4e6032251a6cfc&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:54.626517 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913506a8-6442-4934-85f1-a838970f4abb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:54.714819 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:54.731269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381e65c-0acf-4b7d-b054-f07af04092c8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.735941 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913506a8-6442-4934-85f1-a838970f4abb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.767993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af37987-4656-4271-8f92-4183b72ee8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.768666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af37987-4656-4271-8f92-4183b72ee8d1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.768943 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af37987-4656-4271-8f92-4183b72ee8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.769217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af37987-4656-4271-8f92-4183b72ee8d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.769972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af37987-4656-4271-8f92-4183b72ee8d1 None admin] RBAC: Adding query filter params (interface=admin, service_id=8af331bee91c44fdb18575a4494011d1, region_id=RegionOne) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:54.770285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af37987-4656-4271-8f92-4183b72ee8d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.771911 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af37987-4656-4271-8f92-4183b72ee8d1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.777267 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76418847-cae3-4ad3-87ee-5a48375ef778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.778393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76418847-cae3-4ad3-87ee-5a48375ef778 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.778928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76418847-cae3-4ad3-87ee-5a48375ef778 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.779394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76418847-cae3-4ad3-87ee-5a48375ef778 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.780188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76418847-cae3-4ad3-87ee-5a48375ef778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.781748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76418847-cae3-4ad3-87ee-5a48375ef778 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.783239 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 374/752] 10.222.0.26 () {56 vars in 1249 bytes} [Mon Oct 24 08:35:54 2022] GET /identity/v3/endpoints?service_id=8af331bee91c44fdb18575a4494011d1&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:54.785680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af37987-4656-4271-8f92-4183b72ee8d1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:54.799408 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 379/753] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Oct 24 08:35:54 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:35:54.804627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:54.904856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af37987-4656-4271-8f92-4183b72ee8d1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:54.943970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4595f5-d6f5-4d5b-bf2e-8a5c0223d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:54.944597 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4595f5-d6f5-4d5b-bf2e-8a5c0223d3ac None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:54.944874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4595f5-d6f5-4d5b-bf2e-8a5c0223d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:54.945142 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4595f5-d6f5-4d5b-bf2e-8a5c0223d3ac None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:54.945768 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4595f5-d6f5-4d5b-bf2e-8a5c0223d3ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:54.947301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4595f5-d6f5-4d5b-bf2e-8a5c0223d3ac None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:54.954843 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 375/754] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Oct 24 08:35:54 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:35:55.027299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649deac1-f06c-4a3a-9145-c466cd9ccf2a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:55.027828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649deac1-f06c-4a3a-9145-c466cd9ccf2a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:55.027904 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649deac1-f06c-4a3a-9145-c466cd9ccf2a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:55.029370 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 380/755] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:55.035310 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc9561-143c-4af2-8fa5-9ba888eac38e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:55.035735 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc9561-143c-4af2-8fa5-9ba888eac38e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:55.036107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc9561-143c-4af2-8fa5-9ba888eac38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:55.059216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2ecc9561-143c-4af2-8fa5-9ba888eac38e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:55.060433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2ecc9561-143c-4af2-8fa5-9ba888eac38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:55.134670 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 376/756] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:55 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:55.411039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649deac1-f06c-4a3a-9145-c466cd9ccf2a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:55.512116 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649deac1-f06c-4a3a-9145-c466cd9ccf2a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:55.551233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ccec1-ef8c-4dbc-8687-bdbf21385bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:55.551792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ccec1-ef8c-4dbc-8687-bdbf21385bba None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:55.552052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ccec1-ef8c-4dbc-8687-bdbf21385bba None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:55.552333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ccec1-ef8c-4dbc-8687-bdbf21385bba None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:55.552968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ccec1-ef8c-4dbc-8687-bdbf21385bba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:55.554413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ccec1-ef8c-4dbc-8687-bdbf21385bba None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:55.571301 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.sql.core [None req-2d2ccec1-ef8c-4dbc-8687-bdbf21385bba None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 08:35:55.571301 dv-cinder-master-3-smb devstack@keystone.service[87873]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 24 08:35:55.571301 dv-cinder-master-3-smb devstack@keystone.service[87873]: [parameters: {'user_id': '840eeeee156741138f511d93cde33e43', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 24 08:35:55.571301 dv-cinder-master-3-smb devstack@keystone.service[87873]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87873) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2d2ccec1-ef8c-4dbc-8687-bdbf21385bba None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Oct 24 08:35:55.579186 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:35:55.580170 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application yield resource Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise self.value Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application session.commit() Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 08:35:55.581144 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.flush() Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application compat.raise_( Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 08:35:55.582211 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:35:55.583143 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:35:55.583975 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [parameters: {'user_id': '840eeeee156741138f511d93cde33e43', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:55.584829 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 24 08:35:55.586966 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 24 08:35:55.589545 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 24 08:35:55.589545 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 24 08:35:55.589545 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 08:35:55.589545 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 08:35:55.589545 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 24 08:35:55.589545 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:55.589545 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 381/757] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Oct 24 08:35:55 2022] POST /identity/v3/users => generated 160 bytes in 170 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:55.590412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc9561-143c-4af2-8fa5-9ba888eac38e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:55.599715 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:55.696512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc9561-143c-4af2-8fa5-9ba888eac38e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:55.711326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ba2d1-8109-49e8-ae42-646a2b0db125 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:55.711633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ba2d1-8109-49e8-ae42-646a2b0db125 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:55.711862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ba2d1-8109-49e8-ae42-646a2b0db125 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:55.720233 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 382/758] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:55.721875 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173f1f2-a99a-47b1-b45d-a865f57c34b1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:55.722160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173f1f2-a99a-47b1-b45d-a865f57c34b1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:55.722431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173f1f2-a99a-47b1-b45d-a865f57c34b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:55.731602 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaaa76c-cd72-4537-b244-0f992f6673e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:55.732237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaaa76c-cd72-4537-b244-0f992f6673e6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:55.732524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaaa76c-cd72-4537-b244-0f992f6673e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:55.732788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaaa76c-cd72-4537-b244-0f992f6673e6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:55.736247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:55.741694 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaaa76c-cd72-4537-b244-0f992f6673e6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:55.742214 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1173f1f2-a99a-47b1-b45d-a865f57c34b1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:55.743429 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaaa76c-cd72-4537-b244-0f992f6673e6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:55.744241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1173f1f2-a99a-47b1-b45d-a865f57c34b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-dbaaa76c-cd72-4537-b244-0f992f6673e6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:35:55.756507 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:55.758252 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:55.759597 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 377/759] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:35:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:55.763432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaaa76c-cd72-4537-b244-0f992f6673e6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:55.821228 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 383/760] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:55 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:55.846457 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:55.887166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaaa76c-cd72-4537-b244-0f992f6673e6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:55.921812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48ebec-7e5b-4e37-82d6-20443c5343d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:55.922450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48ebec-7e5b-4e37-82d6-20443c5343d2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:55.922747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48ebec-7e5b-4e37-82d6-20443c5343d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:55.923023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48ebec-7e5b-4e37-82d6-20443c5343d2 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:55.923804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48ebec-7e5b-4e37-82d6-20443c5343d2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:55.924110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48ebec-7e5b-4e37-82d6-20443c5343d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:55.925739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48ebec-7e5b-4e37-82d6-20443c5343d2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:55.962147 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 378/761] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Oct 24 08:35:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:56.117478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173f1f2-a99a-47b1-b45d-a865f57c34b1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:56.220892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173f1f2-a99a-47b1-b45d-a865f57c34b1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:56.255640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e80cce-ef36-4317-9814-e12d8515b492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:56.256287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e80cce-ef36-4317-9814-e12d8515b492 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:56.256567 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e80cce-ef36-4317-9814-e12d8515b492 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:56.256837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e80cce-ef36-4317-9814-e12d8515b492 None admin] PATH_INFO: `/v3/services/image` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:56.257419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e80cce-ef36-4317-9814-e12d8515b492 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:56.258942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e80cce-ef36-4317-9814-e12d8515b492 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:56.261716 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-96e80cce-ef36-4317-9814-e12d8515b492 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:56.269311 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:56.270332 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:56.271204 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 384/762] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Oct 24 08:35:56 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:56.276647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48ebec-7e5b-4e37-82d6-20443c5343d2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:56.377891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48ebec-7e5b-4e37-82d6-20443c5343d2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:56.397538 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:56.412830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258a273-adfa-4228-a472-a69604b7b552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:56.413467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258a273-adfa-4228-a472-a69604b7b552 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:56.413764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258a273-adfa-4228-a472-a69604b7b552 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:56.414077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258a273-adfa-4228-a472-a69604b7b552 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:56.414710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258a273-adfa-4228-a472-a69604b7b552 None admin] RBAC: Adding query filter params (name=image) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:56.415037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258a273-adfa-4228-a472-a69604b7b552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:56.416418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258a273-adfa-4228-a472-a69604b7b552 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:56.421498 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 379/763] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:56.427270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e80cce-ef36-4317-9814-e12d8515b492 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:56.528877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e80cce-ef36-4317-9814-e12d8515b492 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:56.562871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb49240-303f-4d9c-9a1f-cd5e2a287ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:56.563483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb49240-303f-4d9c-9a1f-cd5e2a287ab0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:56.563759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb49240-303f-4d9c-9a1f-cd5e2a287ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:56.564030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb49240-303f-4d9c-9a1f-cd5e2a287ab0 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:56.564646 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb49240-303f-4d9c-9a1f-cd5e2a287ab0 None admin] RBAC: Adding query filter params (type=image) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:56.564948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb49240-303f-4d9c-9a1f-cd5e2a287ab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:56.566307 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb49240-303f-4d9c-9a1f-cd5e2a287ab0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:56.574808 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 385/764] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:56.577657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258a273-adfa-4228-a472-a69604b7b552 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:56.678676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258a273-adfa-4228-a472-a69604b7b552 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:56.713732 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7edd2-8fa5-49b6-9c83-871b6e26cb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:56.714344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7edd2-8fa5-49b6-9c83-871b6e26cb54 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:56.714620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7edd2-8fa5-49b6-9c83-871b6e26cb54 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:56.714916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7edd2-8fa5-49b6-9c83-871b6e26cb54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:56.715614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7edd2-8fa5-49b6-9c83-871b6e26cb54 None admin] RBAC: Adding query filter params (interface=public, service_id=18bec32d73f149c08a58fa06d68ffd42, region_id=RegionOne) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:56.715921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7edd2-8fa5-49b6-9c83-871b6e26cb54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:56.717293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7edd2-8fa5-49b6-9c83-871b6e26cb54 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:56.722392 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 380/765] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Oct 24 08:35:56 2022] GET /identity/v3/endpoints?service_id=18bec32d73f149c08a58fa06d68ffd42&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:56.728340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb49240-303f-4d9c-9a1f-cd5e2a287ab0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:56.791234 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:56.831498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb49240-303f-4d9c-9a1f-cd5e2a287ab0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:56.865746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a9f90-d71c-4d0a-a584-574feba466a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:56.866368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a9f90-d71c-4d0a-a584-574feba466a1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:56.866651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a9f90-d71c-4d0a-a584-574feba466a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:56.866953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a9f90-d71c-4d0a-a584-574feba466a1 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:56.867554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a9f90-d71c-4d0a-a584-574feba466a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:56.868934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a9f90-d71c-4d0a-a584-574feba466a1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:56.878801 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 386/766] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Oct 24 08:35:56 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:35:57.076937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e8e26-1e7e-41d1-a7a9-20796e21098c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:57.077288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e8e26-1e7e-41d1-a7a9-20796e21098c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:57.077401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e8e26-1e7e-41d1-a7a9-20796e21098c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:57.078965 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 381/767] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:57.084240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d758ae4-1d36-442e-b33a-c944f0f0a921 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:57.084525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d758ae4-1d36-442e-b33a-c944f0f0a921 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:57.084795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d758ae4-1d36-442e-b33a-c944f0f0a921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:57.104648 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0d758ae4-1d36-442e-b33a-c944f0f0a921 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:57.105923 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0d758ae4-1d36-442e-b33a-c944f0f0a921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:57.179678 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 387/768] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:57 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:57.226031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b9dbd-62d8-4bfb-a41a-7307c82f7624 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:57.226364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b9dbd-62d8-4bfb-a41a-7307c82f7624 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:57.226537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b9dbd-62d8-4bfb-a41a-7307c82f7624 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:57.228101 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 382/769] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:57.233346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6bb4c-5609-4104-b64c-5c2b1c32d556 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:57.233631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6bb4c-5609-4104-b64c-5c2b1c32d556 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:57.233864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6bb4c-5609-4104-b64c-5c2b1c32d556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:57.252632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-60c6bb4c-5609-4104-b64c-5c2b1c32d556 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:57.253800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-60c6bb4c-5609-4104-b64c-5c2b1c32d556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:57.319882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:57.336354 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 388/770] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:57 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:57.442120 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b9dbd-62d8-4bfb-a41a-7307c82f7624 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:57.496880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:57.550226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b9dbd-62d8-4bfb-a41a-7307c82f7624 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:57.592501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f509a-b02f-434b-95b4-d38d4c29f05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:57.593055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f509a-b02f-434b-95b4-d38d4c29f05d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:57.593317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f509a-b02f-434b-95b4-d38d4c29f05d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:57.593597 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f509a-b02f-434b-95b4-d38d4c29f05d None admin] PATH_INFO: `/v3/services/compute` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:57.594180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f509a-b02f-434b-95b4-d38d4c29f05d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:57.595570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f509a-b02f-434b-95b4-d38d4c29f05d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-038f509a-b02f-434b-95b4-d38d4c29f05d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:57.604562 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:57.605619 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 08:35:57.606466 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:57.607263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6bb4c-5609-4104-b64c-5c2b1c32d556 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:57.607342 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 383/771] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:35:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:57.611745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f509a-b02f-434b-95b4-d38d4c29f05d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:57.710952 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6bb4c-5609-4104-b64c-5c2b1c32d556 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:57.711881 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f509a-b02f-434b-95b4-d38d4c29f05d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:57.725950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:57.747682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3badef-c11f-4053-9b8c-191fa971848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:57.748350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3badef-c11f-4053-9b8c-191fa971848f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:57.748626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3badef-c11f-4053-9b8c-191fa971848f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:57.748895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3badef-c11f-4053-9b8c-191fa971848f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:57.749224 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c5d0a-2f34-4336-8f3e-7d26a287d44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:57.749474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3badef-c11f-4053-9b8c-191fa971848f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:57.749831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c5d0a-2f34-4336-8f3e-7d26a287d44b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:57.750110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c5d0a-2f34-4336-8f3e-7d26a287d44b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:57.750381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c5d0a-2f34-4336-8f3e-7d26a287d44b None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:57.750873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3badef-c11f-4053-9b8c-191fa971848f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:57.751032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c5d0a-2f34-4336-8f3e-7d26a287d44b None admin] RBAC: Adding query filter params (name=compute) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:57.751335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c5d0a-2f34-4336-8f3e-7d26a287d44b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:57.752701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c5d0a-2f34-4336-8f3e-7d26a287d44b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:57.758049 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 384/772] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-0f3badef-c11f-4053-9b8c-191fa971848f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:57.760048 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:57.761171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:35:57.763827 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 389/773] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Oct 24 08:35:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:57.764638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c5d0a-2f34-4336-8f3e-7d26a287d44b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:57.767863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3badef-c11f-4053-9b8c-191fa971848f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:57.832017 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:57.879026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3badef-c11f-4053-9b8c-191fa971848f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:57.879388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c5d0a-2f34-4336-8f3e-7d26a287d44b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:57.912963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b957283-cc47-4c09-8e86-b13b7a48bde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:57.913886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b957283-cc47-4c09-8e86-b13b7a48bde5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:57.914323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b957283-cc47-4c09-8e86-b13b7a48bde5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:57.914783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b957283-cc47-4c09-8e86-b13b7a48bde5 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:57.915464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b957283-cc47-4c09-8e86-b13b7a48bde5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:57.915773 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b957283-cc47-4c09-8e86-b13b7a48bde5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:57.917251 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b957283-cc47-4c09-8e86-b13b7a48bde5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:57.917651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f02b69-23d4-4e07-a73e-af991311f202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:57.918259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f02b69-23d4-4e07-a73e-af991311f202 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:57.918537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f02b69-23d4-4e07-a73e-af991311f202 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:57.918846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f02b69-23d4-4e07-a73e-af991311f202 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:57.919470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f02b69-23d4-4e07-a73e-af991311f202 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:57.919774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f02b69-23d4-4e07-a73e-af991311f202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:57.921135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f02b69-23d4-4e07-a73e-af991311f202 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:57.922543 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 390/774] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:57.926235 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 385/775] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Oct 24 08:35:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:57.928154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b957283-cc47-4c09-8e86-b13b7a48bde5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:57.931695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f02b69-23d4-4e07-a73e-af991311f202 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:57.973684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:58.036131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f02b69-23d4-4e07-a73e-af991311f202 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:58.040524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b957283-cc47-4c09-8e86-b13b7a48bde5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:58.056597 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:58.072227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac1f40-14b3-48f1-88dc-9814631cc436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:58.072834 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac1f40-14b3-48f1-88dc-9814631cc436 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:58.073124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac1f40-14b3-48f1-88dc-9814631cc436 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:58.073398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac1f40-14b3-48f1-88dc-9814631cc436 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:58.074109 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac1f40-14b3-48f1-88dc-9814631cc436 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:58.075297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631be4b4-636e-4905-a007-4d5b5bf72203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:58.075538 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac1f40-14b3-48f1-88dc-9814631cc436 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:58.075907 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631be4b4-636e-4905-a007-4d5b5bf72203 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:58.076189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631be4b4-636e-4905-a007-4d5b5bf72203 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:58.076465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631be4b4-636e-4905-a007-4d5b5bf72203 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:58.077082 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631be4b4-636e-4905-a007-4d5b5bf72203 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:58.077382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631be4b4-636e-4905-a007-4d5b5bf72203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:58.078758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631be4b4-636e-4905-a007-4d5b5bf72203 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:58.088828 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 391/776] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Oct 24 08:35:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:35:58.090401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631be4b4-636e-4905-a007-4d5b5bf72203 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:58.102292 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 386/777] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:35:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:35:58.208720 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631be4b4-636e-4905-a007-4d5b5bf72203 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:58.251781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d217edb-6d27-497c-9918-b9d343395e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:58.252404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d217edb-6d27-497c-9918-b9d343395e81 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:58.252658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d217edb-6d27-497c-9918-b9d343395e81 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:58.252962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d217edb-6d27-497c-9918-b9d343395e81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:58.253640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d217edb-6d27-497c-9918-b9d343395e81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:58.255074 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d217edb-6d27-497c-9918-b9d343395e81 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:58.283024 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 392/778] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:35:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:35:59.022810 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19cf31-6d06-4015-a393-ed9946d2195e None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:59.023365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19cf31-6d06-4015-a393-ed9946d2195e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:59.023365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19cf31-6d06-4015-a393-ed9946d2195e None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:59.024909 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 387/779] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:35:59.030432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e28ee-4786-4f38-9380-03a8f6b28ad0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:59.030738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e28ee-4786-4f38-9380-03a8f6b28ad0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:59.031014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e28ee-4786-4f38-9380-03a8f6b28ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:59.050068 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d39e28ee-4786-4f38-9380-03a8f6b28ad0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:35:59.051276 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d39e28ee-4786-4f38-9380-03a8f6b28ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:35:59.082926 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:59.134751 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 393/780] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:35:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:35:59.391669 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19cf31-6d06-4015-a393-ed9946d2195e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:59.491630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:59.501253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19cf31-6d06-4015-a393-ed9946d2195e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:59.537410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdc227-da4b-4fa9-8bbb-eab19de0f271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:59.537952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdc227-da4b-4fa9-8bbb-eab19de0f271 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:59.538210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdc227-da4b-4fa9-8bbb-eab19de0f271 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:59.538503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdc227-da4b-4fa9-8bbb-eab19de0f271 None admin] PATH_INFO: `/v3/services/image` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:59.539094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdc227-da4b-4fa9-8bbb-eab19de0f271 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:59.540468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdc227-da4b-4fa9-8bbb-eab19de0f271 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-cfcdc227-da4b-4fa9-8bbb-eab19de0f271 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:35:59.549357 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:35:59.550356 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:35:59.555517 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 388/781] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Oct 24 08:35:59 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:35:59.557639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e28ee-4786-4f38-9380-03a8f6b28ad0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:59.656725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e28ee-4786-4f38-9380-03a8f6b28ad0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:59.690841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04a1ce-3fa0-4fad-be93-e876faaf83a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:59.691442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a1ce-3fa0-4fad-be93-e876faaf83a8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:59.691719 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a1ce-3fa0-4fad-be93-e876faaf83a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:59.691943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04a1ce-3fa0-4fad-be93-e876faaf83a8 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:59.692594 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04a1ce-3fa0-4fad-be93-e876faaf83a8 None admin] RBAC: Adding query filter params (name=image) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:59.692889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04a1ce-3fa0-4fad-be93-e876faaf83a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:59.694246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04a1ce-3fa0-4fad-be93-e876faaf83a8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:59.702904 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 394/782] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:59.705130 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdc227-da4b-4fa9-8bbb-eab19de0f271 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:59.807688 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdc227-da4b-4fa9-8bbb-eab19de0f271 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:59.842495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3489098-7887-4a55-a0d7-f0bd13f01b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:59.843133 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3489098-7887-4a55-a0d7-f0bd13f01b11 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:59.843415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3489098-7887-4a55-a0d7-f0bd13f01b11 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:59.843689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3489098-7887-4a55-a0d7-f0bd13f01b11 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:59.844315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3489098-7887-4a55-a0d7-f0bd13f01b11 None admin] RBAC: Adding query filter params (type=image) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:35:59.844613 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3489098-7887-4a55-a0d7-f0bd13f01b11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:59.845984 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3489098-7887-4a55-a0d7-f0bd13f01b11 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:35:59.854752 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 389/783] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:35:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:35:59.857150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04a1ce-3fa0-4fad-be93-e876faaf83a8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:35:59.884503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:35:59.959063 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04a1ce-3fa0-4fad-be93-e876faaf83a8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:35:59.993449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee81cfc-2ab2-49b2-8fc1-e14a49ac5ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:35:59.994069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee81cfc-2ab2-49b2-8fc1-e14a49ac5ed8 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:35:59.994350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee81cfc-2ab2-49b2-8fc1-e14a49ac5ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:35:59.994623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee81cfc-2ab2-49b2-8fc1-e14a49ac5ed8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:35:59.995367 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee81cfc-2ab2-49b2-8fc1-e14a49ac5ed8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:35:59.996748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee81cfc-2ab2-49b2-8fc1-e14a49ac5ed8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:00.022759 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 395/784] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:35:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:36:00.328149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c093399-2df9-49ab-8fbb-5cc11118a7f5 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:00.328771 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c093399-2df9-49ab-8fbb-5cc11118a7f5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:00.328771 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c093399-2df9-49ab-8fbb-5cc11118a7f5 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:00.330050 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 390/785] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:00.336339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840fe20-a9e1-40fa-a78c-2c7529f2f1c6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:00.336628 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840fe20-a9e1-40fa-a78c-2c7529f2f1c6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:00.337165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840fe20-a9e1-40fa-a78c-2c7529f2f1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:00.356605 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3840fe20-a9e1-40fa-a78c-2c7529f2f1c6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:00.357804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3840fe20-a9e1-40fa-a78c-2c7529f2f1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:00.433877 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa8865-0bd7-4cd7-8494-09cb9b724cb1 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:00.434232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa8865-0bd7-4cd7-8494-09cb9b724cb1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:00.434424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa8865-0bd7-4cd7-8494-09cb9b724cb1 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:00.436043 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 391/786] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:36:00.440550 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 396/787] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:00.440985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ead30-b44b-4d9f-b26e-851791b90f62 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:00.441335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ead30-b44b-4d9f-b26e-851791b90f62 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:00.441677 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ead30-b44b-4d9f-b26e-851791b90f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:00.459909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-734ead30-b44b-4d9f-b26e-851791b90f62 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:00.461188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-734ead30-b44b-4d9f-b26e-851791b90f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:00.479525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:00.531349 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 392/788] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:00.701891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840fe20-a9e1-40fa-a78c-2c7529f2f1c6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:00.712440 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:00.796656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ead30-b44b-4d9f-b26e-851791b90f62 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:00.805721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840fe20-a9e1-40fa-a78c-2c7529f2f1c6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:00.842289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e3b04-a688-4af9-af7d-256c1bdf8dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:00.842954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e3b04-a688-4af9-af7d-256c1bdf8dff None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:00.843231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e3b04-a688-4af9-af7d-256c1bdf8dff None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:00.843497 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e3b04-a688-4af9-af7d-256c1bdf8dff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:00.853425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e3b04-a688-4af9-af7d-256c1bdf8dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a04e3b04-a688-4af9-af7d-256c1bdf8dff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:36:00.858178 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:00.859171 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:00.860019 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 397/789] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Oct 24 08:36:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:00.866095 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e3b04-a688-4af9-af7d-256c1bdf8dff None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:00.905325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ead30-b44b-4d9f-b26e-851791b90f62 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:00.937914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:00.945263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98159cfc-e067-49a1-9a3f-14b4bcd4cc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:00.945903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98159cfc-e067-49a1-9a3f-14b4bcd4cc37 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:00.946184 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98159cfc-e067-49a1-9a3f-14b4bcd4cc37 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:00.946454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98159cfc-e067-49a1-9a3f-14b4bcd4cc37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:00.947573 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98159cfc-e067-49a1-9a3f-14b4bcd4cc37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:00.949373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98159cfc-e067-49a1-9a3f-14b4bcd4cc37 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:00.950975 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 393/790] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:36:00.956709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98159cfc-e067-49a1-9a3f-14b4bcd4cc37 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:00.977724 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e3b04-a688-4af9-af7d-256c1bdf8dff None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:00.983343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:01.017131 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd37525-367e-4dec-af64-67517d146882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:01.017955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd37525-367e-4dec-af64-67517d146882 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:01.018419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd37525-367e-4dec-af64-67517d146882 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:01.018691 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd37525-367e-4dec-af64-67517d146882 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:01.019378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd37525-367e-4dec-af64-67517d146882 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:01.019678 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd37525-367e-4dec-af64-67517d146882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:01.021048 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd37525-367e-4dec-af64-67517d146882 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:01.029616 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 398/791] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Oct 24 08:36:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:36:01.035092 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd37525-367e-4dec-af64-67517d146882 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:01.064119 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98159cfc-e067-49a1-9a3f-14b4bcd4cc37 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:01.097258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a6dfe-6f47-43d3-bc9b-883461a99d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:01.097856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a6dfe-6f47-43d3-bc9b-883461a99d8f None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:01.098138 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a6dfe-6f47-43d3-bc9b-883461a99d8f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:01.098406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a6dfe-6f47-43d3-bc9b-883461a99d8f None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:01.099083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a6dfe-6f47-43d3-bc9b-883461a99d8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:01.100536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a6dfe-6f47-43d3-bc9b-883461a99d8f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:01.138752 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 394/792] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:36:00 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:36:01.145669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd37525-367e-4dec-af64-67517d146882 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:01.179366 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:01.185623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22613d-7f88-47f9-afcf-2a594b0d7edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:01.186433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22613d-7f88-47f9-afcf-2a594b0d7edc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:01.186741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22613d-7f88-47f9-afcf-2a594b0d7edc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:01.187010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22613d-7f88-47f9-afcf-2a594b0d7edc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:01.199901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22613d-7f88-47f9-afcf-2a594b0d7edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:01.201448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22613d-7f88-47f9-afcf-2a594b0d7edc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-cf22613d-7f88-47f9-afcf-2a594b0d7edc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:36:01.217676 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:01.218681 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:36:01.219956 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:36:01.219956 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:01.219956 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:01.219956 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 399/793] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:36:01.224690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a6dfe-6f47-43d3-bc9b-883461a99d8f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:01.312664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:01.340470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a6dfe-6f47-43d3-bc9b-883461a99d8f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:01.374864 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6331b-3a34-4437-a493-55c77818aba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:01.375468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6331b-3a34-4437-a493-55c77818aba1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:01.375740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6331b-3a34-4437-a493-55c77818aba1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:01.376006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6331b-3a34-4437-a493-55c77818aba1 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:01.376639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6331b-3a34-4437-a493-55c77818aba1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:01.376937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6331b-3a34-4437-a493-55c77818aba1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:01.378288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6331b-3a34-4437-a493-55c77818aba1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:01.390339 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 395/794] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:01.396298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22613d-7f88-47f9-afcf-2a594b0d7edc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:01.496880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22613d-7f88-47f9-afcf-2a594b0d7edc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:01.531443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19ca21-4494-4787-831f-c252500f18f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:01.532078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19ca21-4494-4787-831f-c252500f18f9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:01.532352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19ca21-4494-4787-831f-c252500f18f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:01.532617 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19ca21-4494-4787-831f-c252500f18f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:01.540725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19ca21-4494-4787-831f-c252500f18f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:01.542221 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19ca21-4494-4787-831f-c252500f18f9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ee19ca21-4494-4787-831f-c252500f18f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:36:01.553162 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:01.554110 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:01.555125 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 400/795] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Oct 24 08:36:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:36:01.561989 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6331b-3a34-4437-a493-55c77818aba1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:01.677306 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6331b-3a34-4437-a493-55c77818aba1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:01.711883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951a18a-cc15-4d18-89d9-9d5e68163d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:01.712488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951a18a-cc15-4d18-89d9-9d5e68163d71 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:01.712767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951a18a-cc15-4d18-89d9-9d5e68163d71 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:01.713039 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951a18a-cc15-4d18-89d9-9d5e68163d71 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:01.713793 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951a18a-cc15-4d18-89d9-9d5e68163d71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:01.714094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951a18a-cc15-4d18-89d9-9d5e68163d71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:01.715559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951a18a-cc15-4d18-89d9-9d5e68163d71 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:01.717227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:01.734900 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 396/796] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Oct 24 08:36:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:01.740546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19ca21-4494-4787-831f-c252500f18f9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:01.783712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:01.841516 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19ca21-4494-4787-831f-c252500f18f9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:01.874681 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d3206-3b63-441f-8dea-176b6b97e6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:01.875341 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d3206-3b63-441f-8dea-176b6b97e6cc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:01.875619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d3206-3b63-441f-8dea-176b6b97e6cc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:01.875894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d3206-3b63-441f-8dea-176b6b97e6cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:01.886520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d3206-3b63-441f-8dea-176b6b97e6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:01.888007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d3206-3b63-441f-8dea-176b6b97e6cc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-6f3d3206-3b63-441f-8dea-176b6b97e6cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:36:01.902189 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:01.903206 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:36:01.904097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:36:01.904097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:01.904097 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:01.904940 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 401/797] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:01.909154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951a18a-cc15-4d18-89d9-9d5e68163d71 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:02.012520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951a18a-cc15-4d18-89d9-9d5e68163d71 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:02.046352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666c987-3a2f-41c6-a0e5-baa7160b4329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:02.046924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666c987-3a2f-41c6-a0e5-baa7160b4329 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:02.047200 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666c987-3a2f-41c6-a0e5-baa7160b4329 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:02.047469 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666c987-3a2f-41c6-a0e5-baa7160b4329 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:02.048117 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666c987-3a2f-41c6-a0e5-baa7160b4329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:02.048420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666c987-3a2f-41c6-a0e5-baa7160b4329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:02.049846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666c987-3a2f-41c6-a0e5-baa7160b4329 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:02.064378 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 397/798] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:02.070541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d3206-3b63-441f-8dea-176b6b97e6cc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:02.171051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:02.175002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3bac9-563c-4ab2-929e-0b86c56a93b2 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:02.175530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3bac9-563c-4ab2-929e-0b86c56a93b2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:02.176124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3bac9-563c-4ab2-929e-0b86c56a93b2 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:02.177786 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 398/799] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:02.179031 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d3206-3b63-441f-8dea-176b6b97e6cc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:02.183000 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67992df-4c79-4347-a84b-1a3012f825ec None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:02.183517 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67992df-4c79-4347-a84b-1a3012f825ec None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:02.184002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67992df-4c79-4347-a84b-1a3012f825ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:02.206315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b67992df-4c79-4347-a84b-1a3012f825ec None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:02.207572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b67992df-4c79-4347-a84b-1a3012f825ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:02.214394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0342a3-117d-49e1-aebe-8666858f7613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:02.215039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0342a3-117d-49e1-aebe-8666858f7613 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:02.215580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0342a3-117d-49e1-aebe-8666858f7613 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:02.215580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0342a3-117d-49e1-aebe-8666858f7613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:02.222192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0342a3-117d-49e1-aebe-8666858f7613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:02.224042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0342a3-117d-49e1-aebe-8666858f7613 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-4b0342a3-117d-49e1-aebe-8666858f7613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:36:02.235131 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:02.236004 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:02.236897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:36:02.236897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:36:02.236897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:36:02.236897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:36:02.236897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:36:02.236897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:36:02.236897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:36:02.236897 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:02.236897 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 402/800] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:36:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:02.242097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0342a3-117d-49e1-aebe-8666858f7613 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:02.259668 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:02.291885 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 399/801] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:02.346636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0342a3-117d-49e1-aebe-8666858f7613 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:02.379775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd0d0a-bb1f-49b7-b447-7cfd29e3f47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:02.380421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd0d0a-bb1f-49b7-b447-7cfd29e3f47e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:02.380653 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd0d0a-bb1f-49b7-b447-7cfd29e3f47e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:02.380914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd0d0a-bb1f-49b7-b447-7cfd29e3f47e None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:02.381596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd0d0a-bb1f-49b7-b447-7cfd29e3f47e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:02.381898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd0d0a-bb1f-49b7-b447-7cfd29e3f47e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:02.383356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd0d0a-bb1f-49b7-b447-7cfd29e3f47e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:02.395245 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 403/802] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:36:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:02.401224 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67992df-4c79-4347-a84b-1a3012f825ec None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:02.502461 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67992df-4c79-4347-a84b-1a3012f825ec None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:02.536831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60a6e0-0bcf-476e-9973-cf75f0dfc7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:02.537502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60a6e0-0bcf-476e-9973-cf75f0dfc7ef None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:02.537782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60a6e0-0bcf-476e-9973-cf75f0dfc7ef None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:02.538056 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60a6e0-0bcf-476e-9973-cf75f0dfc7ef None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/f257d9b665714bff8b4ceb4fc5dc8387/roles/1c0b192e409a435a98f939e77554596d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:02.540238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60a6e0-0bcf-476e-9973-cf75f0dfc7ef None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=f257d9b665714bff8b4ceb4fc5dc8387, role_id=1c0b192e409a435a98f939e77554596d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:02.541740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60a6e0-0bcf-476e-9973-cf75f0dfc7ef None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:02.554797 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd0d0a-bb1f-49b7-b447-7cfd29e3f47e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:02.555991 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 400/803] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:36:02 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/f257d9b665714bff8b4ceb4fc5dc8387/roles/1c0b192e409a435a98f939e77554596d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:36:02.572713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:02.664632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd0d0a-bb1f-49b7-b447-7cfd29e3f47e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:02.700296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:02.700928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:02.701210 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:02.701448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:02.702133 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:02.703559 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:02.712407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.sql.core [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 08:36:02.712407 dv-cinder-master-3-smb devstack@keystone.service[87873]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 08:36:02.712407 dv-cinder-master-3-smb devstack@keystone.service[87873]: [parameters: {'id': '0e376b23295b467bbd13d116d9dbc564', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 08:36:02.712407 dv-cinder-master-3-smb devstack@keystone.service[87873]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87873) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:36:02.720025 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 08:36:02.720969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application yield resource Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise self.value Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application session.commit() Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 08:36:02.722055 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.flush() Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application compat.raise_( Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 08:36:02.723197 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 08:36:02.724117 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 08:36:02.724947 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [parameters: {'id': '0e376b23295b467bbd13d116d9dbc564', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 08:36:02.725845 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:02.726634 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:02.729924 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 404/804] 10.222.0.26 () {60 vars in 1137 bytes} [Mon Oct 24 08:36:02 2022] POST /identity/v3/domains => generated 190 bytes in 167 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:02.731468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60a6e0-0bcf-476e-9973-cf75f0dfc7ef None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:02.779203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:02.837333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60a6e0-0bcf-476e-9973-cf75f0dfc7ef None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:02.872174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62c0b7-0eb2-4993-a129-5f7b755f1a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:02.872935 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62c0b7-0eb2-4993-a129-5f7b755f1a9d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:02.873298 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62c0b7-0eb2-4993-a129-5f7b755f1a9d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:02.873643 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62c0b7-0eb2-4993-a129-5f7b755f1a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:02.883916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62c0b7-0eb2-4993-a129-5f7b755f1a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:02.885509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62c0b7-0eb2-4993-a129-5f7b755f1a9d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-8d62c0b7-0eb2-4993-a129-5f7b755f1a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:36:02.900207 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:02.901389 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:36:02.902521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:36:02.902521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:02.902521 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:36:02.903325 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 401/805] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:02.909249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:02.988902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:03.012042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ad2b7-4a7f-4d81-b623-09ac8ab51a38 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:03.047549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed7fc7-c966-4cdc-a642-13ba79336009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:03.048260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed7fc7-c966-4cdc-a642-13ba79336009 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:03.048620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed7fc7-c966-4cdc-a642-13ba79336009 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:03.048959 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed7fc7-c966-4cdc-a642-13ba79336009 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:03.049670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed7fc7-c966-4cdc-a642-13ba79336009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:03.050039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed7fc7-c966-4cdc-a642-13ba79336009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:03.051516 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed7fc7-c966-4cdc-a642-13ba79336009 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:03.063209 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 405/806] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:03.335304 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea997c-ede8-4458-bbae-632f51167755 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:03.335747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea997c-ede8-4458-bbae-632f51167755 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:03.336112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea997c-ede8-4458-bbae-632f51167755 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:03.337712 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 402/807] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:03.343452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17be96-4eec-4915-85ec-8e23cf8c6d3d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:03.343742 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17be96-4eec-4915-85ec-8e23cf8c6d3d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:03.344016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17be96-4eec-4915-85ec-8e23cf8c6d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:03.364102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ab17be96-4eec-4915-85ec-8e23cf8c6d3d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:03.365303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ab17be96-4eec-4915-85ec-8e23cf8c6d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:03.431153 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 406/808] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:03.695063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea997c-ede8-4458-bbae-632f51167755 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:03.792923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:03.801926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea997c-ede8-4458-bbae-632f51167755 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:03.835249 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c585f20-2477-472c-900f-f8139c12e392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:03.835871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c585f20-2477-472c-900f-f8139c12e392 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:03.836146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c585f20-2477-472c-900f-f8139c12e392 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:03.836412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c585f20-2477-472c-900f-f8139c12e392 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:03.837578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c585f20-2477-472c-900f-f8139c12e392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:03.839038 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c585f20-2477-472c-900f-f8139c12e392 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:03.840583 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 403/809] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:03.846564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17be96-4eec-4915-85ec-8e23cf8c6d3d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:03.911212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:03.946010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17be96-4eec-4915-85ec-8e23cf8c6d3d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:03.983340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ffbe5-9b6c-42a7-9e3f-3d6bf0503f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:03.983954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ffbe5-9b6c-42a7-9e3f-3d6bf0503f11 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:03.984246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ffbe5-9b6c-42a7-9e3f-3d6bf0503f11 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:03.984499 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ffbe5-9b6c-42a7-9e3f-3d6bf0503f11 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:03.985152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ffbe5-9b6c-42a7-9e3f-3d6bf0503f11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:03.986588 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ffbe5-9b6c-42a7-9e3f-3d6bf0503f11 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:04.018763 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 407/810] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:36:03 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:36:04.692145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d55e-0ea9-4eae-a754-86a8a0273e6c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:04.692418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d55e-0ea9-4eae-a754-86a8a0273e6c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:04.692679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d55e-0ea9-4eae-a754-86a8a0273e6c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:04.694180 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 404/811] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:04.699422 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047367c-6114-48a7-8e9b-37c6978ae90d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:04.699707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047367c-6114-48a7-8e9b-37c6978ae90d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:04.699982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047367c-6114-48a7-8e9b-37c6978ae90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:04.720323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5047367c-6114-48a7-8e9b-37c6978ae90d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:04.721526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5047367c-6114-48a7-8e9b-37c6978ae90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:04.789909 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 408/812] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:05.056924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6d55e-0ea9-4eae-a754-86a8a0273e6c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:05.165942 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6d55e-0ea9-4eae-a754-86a8a0273e6c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:05.200161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473e666-7a49-4a31-9306-db5ef7a83ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:05.200777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473e666-7a49-4a31-9306-db5ef7a83ba8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:05.201052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473e666-7a49-4a31-9306-db5ef7a83ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:05.201319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473e666-7a49-4a31-9306-db5ef7a83ba8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:05.212264 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473e666-7a49-4a31-9306-db5ef7a83ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3473e666-7a49-4a31-9306-db5ef7a83ba8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:36:05.217102 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:05.218120 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 08:36:05.219082 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:36:05.223087 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 405/813] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Oct 24 08:36:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:05.224327 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047367c-6114-48a7-8e9b-37c6978ae90d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:05.279903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:05.325332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047367c-6114-48a7-8e9b-37c6978ae90d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:05.358168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26daed0-406d-49ca-8de5-82dc938306d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:05.358786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26daed0-406d-49ca-8de5-82dc938306d8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:05.359085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26daed0-406d-49ca-8de5-82dc938306d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:05.359347 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26daed0-406d-49ca-8de5-82dc938306d8 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:05.359995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26daed0-406d-49ca-8de5-82dc938306d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:05.360275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26daed0-406d-49ca-8de5-82dc938306d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:05.361628 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26daed0-406d-49ca-8de5-82dc938306d8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:05.369404 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 409/814] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Oct 24 08:36:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:05.377857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473e666-7a49-4a31-9306-db5ef7a83ba8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:05.403261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:05.478383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473e666-7a49-4a31-9306-db5ef7a83ba8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:05.510340 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01aab6-bf6a-4ebe-8f2b-bd6a5e1b0f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:05.510995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01aab6-bf6a-4ebe-8f2b-bd6a5e1b0f8c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:05.511273 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01aab6-bf6a-4ebe-8f2b-bd6a5e1b0f8c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:05.511544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01aab6-bf6a-4ebe-8f2b-bd6a5e1b0f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:05.521753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01aab6-bf6a-4ebe-8f2b-bd6a5e1b0f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:05.523229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01aab6-bf6a-4ebe-8f2b-bd6a5e1b0f8c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-6c01aab6-bf6a-4ebe-8f2b-bd6a5e1b0f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:36:05.537654 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:05.538638 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:36:05.539566 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:36:05.539566 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:05.539566 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:36:05.539566 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 406/815] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:05.544641 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26daed0-406d-49ca-8de5-82dc938306d8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:05.656013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26daed0-406d-49ca-8de5-82dc938306d8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:05.672807 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:05.692711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cbd5e-9bca-483f-b040-a905dd40bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:05.693322 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cbd5e-9bca-483f-b040-a905dd40bac4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:05.693601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cbd5e-9bca-483f-b040-a905dd40bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:05.693867 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cbd5e-9bca-483f-b040-a905dd40bac4 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:05.694502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cbd5e-9bca-483f-b040-a905dd40bac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:05.694828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cbd5e-9bca-483f-b040-a905dd40bac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:05.696183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cbd5e-9bca-483f-b040-a905dd40bac4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:05.707746 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 410/816] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:05.713599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01aab6-bf6a-4ebe-8f2b-bd6a5e1b0f8c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:05.790174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:05.815430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01aab6-bf6a-4ebe-8f2b-bd6a5e1b0f8c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:05.847648 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7fe22-57e6-4f81-8203-296a0e5fe67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:05.848265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7fe22-57e6-4f81-8203-296a0e5fe67d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:05.848542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7fe22-57e6-4f81-8203-296a0e5fe67d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:05.848812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7fe22-57e6-4f81-8203-296a0e5fe67d None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:05.856361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7fe22-57e6-4f81-8203-296a0e5fe67d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:05.857777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7fe22-57e6-4f81-8203-296a0e5fe67d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e6e7fe22-57e6-4f81-8203-296a0e5fe67d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:36:05.868883 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:05.870014 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:36:05.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 407/817] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Oct 24 08:36:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:36:05.876428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cbd5e-9bca-483f-b040-a905dd40bac4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:05.975182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cbd5e-9bca-483f-b040-a905dd40bac4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:06.008657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc0810-22e3-4b59-87ec-5782a7ca1d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:06.009257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc0810-22e3-4b59-87ec-5782a7ca1d50 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.009531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc0810-22e3-4b59-87ec-5782a7ca1d50 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.009797 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc0810-22e3-4b59-87ec-5782a7ca1d50 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.010567 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc0810-22e3-4b59-87ec-5782a7ca1d50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:06.010894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc0810-22e3-4b59-87ec-5782a7ca1d50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:06.012326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc0810-22e3-4b59-87ec-5782a7ca1d50 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:06.029244 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 411/818] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Oct 24 08:36:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:36:06.035509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7fe22-57e6-4f81-8203-296a0e5fe67d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.139586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7fe22-57e6-4f81-8203-296a0e5fe67d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:06.153224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c25bf-48af-4c83-910f-ea59218820e4 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.153510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c25bf-48af-4c83-910f-ea59218820e4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.153783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c25bf-48af-4c83-910f-ea59218820e4 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.155207 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 412/819] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:06.160137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c276d1-29a3-4f38-9783-c58d5cd7c86b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.160417 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c276d1-29a3-4f38-9783-c58d5cd7c86b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.160697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c276d1-29a3-4f38-9783-c58d5cd7c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.172437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a1203-0c28-4c81-8f05-212c9051172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:06.173035 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a1203-0c28-4c81-8f05-212c9051172f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.173312 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a1203-0c28-4c81-8f05-212c9051172f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.173581 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a1203-0c28-4c81-8f05-212c9051172f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.176159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:06.179480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-49c276d1-29a3-4f38-9783-c58d5cd7c86b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:06.180675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-49c276d1-29a3-4f38-9783-c58d5cd7c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:06.206918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:06.239767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a1203-0c28-4c81-8f05-212c9051172f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:06.241094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a1203-0c28-4c81-8f05-212c9051172f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:06.247680 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 413/820] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:06.340962 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 408/821] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:36:06.346829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c276d1-29a3-4f38-9783-c58d5cd7c86b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.447474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c276d1-29a3-4f38-9783-c58d5cd7c86b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:06.481401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6db4a8-6e88-439d-85aa-3876cafe4702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:06.482020 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6db4a8-6e88-439d-85aa-3876cafe4702 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.482292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6db4a8-6e88-439d-85aa-3876cafe4702 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.482556 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6db4a8-6e88-439d-85aa-3876cafe4702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.492524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6db4a8-6e88-439d-85aa-3876cafe4702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:06.493902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6db4a8-6e88-439d-85aa-3876cafe4702 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:06.507666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a1203-0c28-4c81-8f05-212c9051172f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-3b6db4a8-6e88-439d-85aa-3876cafe4702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:36:06.508419 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:06.509421 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:36:06.510312 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:36:06.510312 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:36:06.510312 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:06.510886 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 414/822] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:06.515495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6db4a8-6e88-439d-85aa-3876cafe4702 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.537412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:06.547833 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:06.614133 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a1203-0c28-4c81-8f05-212c9051172f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:06.620553 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6db4a8-6e88-439d-85aa-3876cafe4702 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:06.648455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eaec99-6e1d-4156-9a74-4c831e4dd816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:06.649147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaec99-6e1d-4156-9a74-4c831e4dd816 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.649422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaec99-6e1d-4156-9a74-4c831e4dd816 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.649685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaec99-6e1d-4156-9a74-4c831e4dd816 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.653363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abb6d7-cc58-4c1e-a4a8-a26c7c49b8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:06.653966 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abb6d7-cc58-4c1e-a4a8-a26c7c49b8e1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.654246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abb6d7-cc58-4c1e-a4a8-a26c7c49b8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.654514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abb6d7-cc58-4c1e-a4a8-a26c7c49b8e1 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.655189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abb6d7-cc58-4c1e-a4a8-a26c7c49b8e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:06.655489 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abb6d7-cc58-4c1e-a4a8-a26c7c49b8e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:06.656848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abb6d7-cc58-4c1e-a4a8-a26c7c49b8e1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:06.660044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eaec99-6e1d-4156-9a74-4c831e4dd816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-b8eaec99-6e1d-4156-9a74-4c831e4dd816 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:36:06.665037 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:06.665995 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:36:06.667128 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 409/823] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:06.668754 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 415/824] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:06.672201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eaec99-6e1d-4156-9a74-4c831e4dd816 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.673732 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abb6d7-cc58-4c1e-a4a8-a26c7c49b8e1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.747612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:06.763252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:06.770472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abb6d7-cc58-4c1e-a4a8-a26c7c49b8e1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:06.770645 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eaec99-6e1d-4156-9a74-4c831e4dd816 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:06.803432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651d4f-434a-47f4-b7db-e181ad92c528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:06.803646 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e9c8f-51ed-40b5-aea4-02da06900c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:06.804023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651d4f-434a-47f4-b7db-e181ad92c528 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.804124 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e9c8f-51ed-40b5-aea4-02da06900c94 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.804298 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651d4f-434a-47f4-b7db-e181ad92c528 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.804392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e9c8f-51ed-40b5-aea4-02da06900c94 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.804565 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651d4f-434a-47f4-b7db-e181ad92c528 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.804657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e9c8f-51ed-40b5-aea4-02da06900c94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.805220 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55651d4f-434a-47f4-b7db-e181ad92c528 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:06.805521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55651d4f-434a-47f4-b7db-e181ad92c528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:06.806945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55651d4f-434a-47f4-b7db-e181ad92c528 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:06.811303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e9c8f-51ed-40b5-aea4-02da06900c94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:06.812747 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e9c8f-51ed-40b5-aea4-02da06900c94 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:06.815188 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 410/825] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:06.820930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651d4f-434a-47f4-b7db-e181ad92c528 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-325e9c8f-51ed-40b5-aea4-02da06900c94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:36:06.823922 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:06.824969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:06.826057 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:36:06.826057 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:36:06.826057 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:36:06.826057 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:36:06.826057 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:36:06.826057 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:36:06.826057 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:36:06.826057 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:06.826057 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 416/826] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:36:06.830681 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e9c8f-51ed-40b5-aea4-02da06900c94 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.919831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651d4f-434a-47f4-b7db-e181ad92c528 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:06.927700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e9c8f-51ed-40b5-aea4-02da06900c94 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:06.953341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c662581-4d17-43bc-a04f-f327ebebbc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:06.954071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c662581-4d17-43bc-a04f-f327ebebbc87 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.954409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c662581-4d17-43bc-a04f-f327ebebbc87 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.954758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c662581-4d17-43bc-a04f-f327ebebbc87 None admin] PATH_INFO: `/v3/users/7aaf4ca40f904baba338bbb3bd28bdf0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.959589 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f81a3-45da-4cab-af68-0be236bbd027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:06.960182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f81a3-45da-4cab-af68-0be236bbd027 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:06.960456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f81a3-45da-4cab-af68-0be236bbd027 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:06.960722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f81a3-45da-4cab-af68-0be236bbd027 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:06.961402 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f81a3-45da-4cab-af68-0be236bbd027 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:06.961704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f81a3-45da-4cab-af68-0be236bbd027 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:06.963149 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f81a3-45da-4cab-af68-0be236bbd027 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:06.965566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:06.968593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c662581-4d17-43bc-a04f-f327ebebbc87 None admin] RBAC: Authorizing `identity:get_user(user_id=7aaf4ca40f904baba338bbb3bd28bdf0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:06.970013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c662581-4d17-43bc-a04f-f327ebebbc87 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:06.972132 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 411/827] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/users/7aaf4ca40f904baba338bbb3bd28bdf0 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:36:06.978177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c662581-4d17-43bc-a04f-f327ebebbc87 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.978344 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 417/828] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:36:06.984567 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f81a3-45da-4cab-af68-0be236bbd027 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:06.994899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:07.080059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c662581-4d17-43bc-a04f-f327ebebbc87 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:07.084963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f81a3-45da-4cab-af68-0be236bbd027 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:07.114085 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c85ce-3d0d-43ff-9ace-41cb56fbeef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:07.114705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c85ce-3d0d-43ff-9ace-41cb56fbeef3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:07.115004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c85ce-3d0d-43ff-9ace-41cb56fbeef3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:07.115277 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c85ce-3d0d-43ff-9ace-41cb56fbeef3 None admin] PATH_INFO: `/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:07.116348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c85ce-3d0d-43ff-9ace-41cb56fbeef3 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a2074e9f454558bc2d3ed1e3ebd255)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:07.117127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aed19-54cc-43ba-b235-6041c140010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:07.117730 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aed19-54cc-43ba-b235-6041c140010f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:07.117837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c85ce-3d0d-43ff-9ace-41cb56fbeef3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:07.118007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aed19-54cc-43ba-b235-6041c140010f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:07.118275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aed19-54cc-43ba-b235-6041c140010f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:07.119050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aed19-54cc-43ba-b235-6041c140010f None admin] RBAC: Adding query filter params (role.id=1c0b192e409a435a98f939e77554596d, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=f257d9b665714bff8b4ceb4fc5dc8387) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:07.119360 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aed19-54cc-43ba-b235-6041c140010f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:07.119496 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 412/829] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:07.120827 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aed19-54cc-43ba-b235-6041c140010f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:07.125290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c85ce-3d0d-43ff-9ace-41cb56fbeef3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:07.127003 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 418/830] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:36:06 2022] GET /identity/v3/role_assignments?user.id=f257d9b665714bff8b4ceb4fc5dc8387&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=1c0b192e409a435a98f939e77554596d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:07.209508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:07.231187 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c85ce-3d0d-43ff-9ace-41cb56fbeef3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:07.272139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162d954-600f-46f7-a5e9-804b8b7eedfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:07.272817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162d954-600f-46f7-a5e9-804b8b7eedfd None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:07.273093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162d954-600f-46f7-a5e9-804b8b7eedfd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:07.273366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162d954-600f-46f7-a5e9-804b8b7eedfd None admin] PATH_INFO: `/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255/users/7aaf4ca40f904baba338bbb3bd28bdf0/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:07.275417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162d954-600f-46f7-a5e9-804b8b7eedfd None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a2074e9f454558bc2d3ed1e3ebd255, user_id=7aaf4ca40f904baba338bbb3bd28bdf0, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:07.276918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162d954-600f-46f7-a5e9-804b8b7eedfd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:07.295995 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 413/831] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:36:07 2022] PUT /identity/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255/users/7aaf4ca40f904baba338bbb3bd28bdf0/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:36:09.401439 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b106c-a10a-4fb0-8f81-f7c48ab1e7e6 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:09.402284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b106c-a10a-4fb0-8f81-f7c48ab1e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:09.402284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b106c-a10a-4fb0-8f81-f7c48ab1e7e6 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:09.404499 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 419/832] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:09.415841 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bfcf1a-9a27-4aaf-b8a4-4772bfb90bb3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:09.416280 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bfcf1a-9a27-4aaf-b8a4-4772bfb90bb3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:09.416391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bfcf1a-9a27-4aaf-b8a4-4772bfb90bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:09.441186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-83bfcf1a-9a27-4aaf-b8a4-4772bfb90bb3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:09.442366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-83bfcf1a-9a27-4aaf-b8a4-4772bfb90bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:09.511056 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 414/833] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:09.778823 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b106c-a10a-4fb0-8f81-f7c48ab1e7e6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:09.853425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:09.894047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b106c-a10a-4fb0-8f81-f7c48ab1e7e6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:09.929300 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24d929-44ee-4e3e-9389-b6455a3fb6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:09.929919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24d929-44ee-4e3e-9389-b6455a3fb6ee None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:09.930198 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24d929-44ee-4e3e-9389-b6455a3fb6ee None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:09.930467 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24d929-44ee-4e3e-9389-b6455a3fb6ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:09.941185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24d929-44ee-4e3e-9389-b6455a3fb6ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-5f24d929-44ee-4e3e-9389-b6455a3fb6ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:36:09.945940 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:09.947026 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:09.948074 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 420/834] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:36:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:09.954302 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bfcf1a-9a27-4aaf-b8a4-4772bfb90bb3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:10.031770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:10.061925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bfcf1a-9a27-4aaf-b8a4-4772bfb90bb3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:10.096379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfe0bf-c57c-499f-8724-a308fbf6c128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:10.096968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfe0bf-c57c-499f-8724-a308fbf6c128 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:10.097241 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfe0bf-c57c-499f-8724-a308fbf6c128 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:10.097508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfe0bf-c57c-499f-8724-a308fbf6c128 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:10.098139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfe0bf-c57c-499f-8724-a308fbf6c128 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:10.098436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfe0bf-c57c-499f-8724-a308fbf6c128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:10.099858 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfe0bf-c57c-499f-8724-a308fbf6c128 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:10.107830 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 415/835] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:36:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:10.117409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24d929-44ee-4e3e-9389-b6455a3fb6ee None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:10.219390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24d929-44ee-4e3e-9389-b6455a3fb6ee None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:10.240587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:10.255642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260432-42ba-415b-8b3c-692dcd7f00df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:10.256253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260432-42ba-415b-8b3c-692dcd7f00df None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:10.256525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260432-42ba-415b-8b3c-692dcd7f00df None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:10.256791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260432-42ba-415b-8b3c-692dcd7f00df None admin] PATH_INFO: `/v3/users/7aaf4ca40f904baba338bbb3bd28bdf0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:10.257954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07260432-42ba-415b-8b3c-692dcd7f00df None admin] RBAC: Authorizing `identity:get_user(user_id=7aaf4ca40f904baba338bbb3bd28bdf0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:10.259423 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07260432-42ba-415b-8b3c-692dcd7f00df None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:10.260982 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 421/836] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:10 2022] GET /identity/v3/users/7aaf4ca40f904baba338bbb3bd28bdf0 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:10.266902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfe0bf-c57c-499f-8724-a308fbf6c128 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:10.371326 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfe0bf-c57c-499f-8724-a308fbf6c128 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:10.405154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e947b77-9dc8-4f7b-aa6d-f5a6e95298e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:10.405740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e947b77-9dc8-4f7b-aa6d-f5a6e95298e5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:10.406015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e947b77-9dc8-4f7b-aa6d-f5a6e95298e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:10.406282 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e947b77-9dc8-4f7b-aa6d-f5a6e95298e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:10.409088 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:10.479537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e947b77-9dc8-4f7b-aa6d-f5a6e95298e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:10.481014 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e947b77-9dc8-4f7b-aa6d-f5a6e95298e5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:10.584856 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 416/837] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:36:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:36:10.591156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260432-42ba-415b-8b3c-692dcd7f00df None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:10.691276 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260432-42ba-415b-8b3c-692dcd7f00df None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:10.725658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada05d71-f364-4553-a17f-548284fbcd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:10.726272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada05d71-f364-4553-a17f-548284fbcd9e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:10.726545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada05d71-f364-4553-a17f-548284fbcd9e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:10.726843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada05d71-f364-4553-a17f-548284fbcd9e None admin] PATH_INFO: `/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:10.727988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada05d71-f364-4553-a17f-548284fbcd9e None admin] RBAC: Authorizing `identity:get_project(project_id=e4a2074e9f454558bc2d3ed1e3ebd255)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:10.729512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada05d71-f364-4553-a17f-548284fbcd9e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:10.731141 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 422/838] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:36:10 2022] GET /identity/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:10.738124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e947b77-9dc8-4f7b-aa6d-f5a6e95298e5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:10.780852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:10.851620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e947b77-9dc8-4f7b-aa6d-f5a6e95298e5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:10.886012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814e002-aad0-4bf0-bd3b-a8f0065563dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:10.886598 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814e002-aad0-4bf0-bd3b-a8f0065563dc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:10.886904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814e002-aad0-4bf0-bd3b-a8f0065563dc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:10.887176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814e002-aad0-4bf0-bd3b-a8f0065563dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:10.887885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814e002-aad0-4bf0-bd3b-a8f0065563dc None admin] RBAC: Adding query filter params (role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.project.id=e4a2074e9f454558bc2d3ed1e3ebd255, user.id=7aaf4ca40f904baba338bbb3bd28bdf0) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:10.888189 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814e002-aad0-4bf0-bd3b-a8f0065563dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:10.889617 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814e002-aad0-4bf0-bd3b-a8f0065563dc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:10.895391 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 417/839] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:36:10 2022] GET /identity/v3/role_assignments?user.id=7aaf4ca40f904baba338bbb3bd28bdf0&scope.project.id=e4a2074e9f454558bc2d3ed1e3ebd255&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:13.028741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b0d0e-f6a5-415a-bfb7-1248004c5276 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:13.029596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b0d0e-f6a5-415a-bfb7-1248004c5276 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:13.029596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b0d0e-f6a5-415a-bfb7-1248004c5276 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:13.032013 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 423/840] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:13.037901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170493f-cee1-49f4-850c-aa231898a04a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:13.038318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170493f-cee1-49f4-850c-aa231898a04a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:13.038405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170493f-cee1-49f4-850c-aa231898a04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:13.058462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2170493f-cee1-49f4-850c-aa231898a04a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:13.059663 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2170493f-cee1-49f4-850c-aa231898a04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:13.126952 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 418/841] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:13.394113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b0d0e-f6a5-415a-bfb7-1248004c5276 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:13.461959 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:13.508881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b0d0e-f6a5-415a-bfb7-1248004c5276 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:13.543305 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37128f64-8333-4564-a050-c28300897073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:13.543863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37128f64-8333-4564-a050-c28300897073 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:13.544142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37128f64-8333-4564-a050-c28300897073 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:13.544414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37128f64-8333-4564-a050-c28300897073 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:13.545573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37128f64-8333-4564-a050-c28300897073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:13.547039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37128f64-8333-4564-a050-c28300897073 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:13.548579 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 424/842] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:13.555145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170493f-cee1-49f4-850c-aa231898a04a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:13.590345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:13.663261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170493f-cee1-49f4-850c-aa231898a04a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:13.820163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea9fb7-09aa-49d9-bcee-e0b6e255fecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:13.820661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea9fb7-09aa-49d9-bcee-e0b6e255fecc None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:13.820939 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea9fb7-09aa-49d9-bcee-e0b6e255fecc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:13.821210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea9fb7-09aa-49d9-bcee-e0b6e255fecc None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:13.821855 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea9fb7-09aa-49d9-bcee-e0b6e255fecc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:13.823327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea9fb7-09aa-49d9-bcee-e0b6e255fecc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:13.847591 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 419/843] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:36:13 2022] POST /identity/v3/users => generated 286 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:36:15.994835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce58c93-5e56-496e-8128-d86b73f21cd5 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:15.995409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce58c93-5e56-496e-8128-d86b73f21cd5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:15.995409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce58c93-5e56-496e-8128-d86b73f21cd5 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:15.996767 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 425/844] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:16.003261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e67e13-97f5-4735-b14f-1e3e0f564f62 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:16.003947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e67e13-97f5-4735-b14f-1e3e0f564f62 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:16.004154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e67e13-97f5-4735-b14f-1e3e0f564f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:16.026857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c2e67e13-97f5-4735-b14f-1e3e0f564f62 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:16.028034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c2e67e13-97f5-4735-b14f-1e3e0f564f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:16.094769 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 420/845] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:16.367856 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce58c93-5e56-496e-8128-d86b73f21cd5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:16.475214 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce58c93-5e56-496e-8128-d86b73f21cd5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:16.499507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:16.514188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbd1b5-0052-4b51-9895-6965349a7d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:16.514845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbd1b5-0052-4b51-9895-6965349a7d84 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:16.515132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbd1b5-0052-4b51-9895-6965349a7d84 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:16.515401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbd1b5-0052-4b51-9895-6965349a7d84 None admin] PATH_INFO: `/v3/roles/5cd206a783e44ac2829c7661b34bb6a7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:16.516843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbd1b5-0052-4b51-9895-6965349a7d84 None admin] RBAC: Authorizing `identity:get_role(role_id=5cd206a783e44ac2829c7661b34bb6a7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:16.518216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbd1b5-0052-4b51-9895-6965349a7d84 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:16.519560 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 426/846] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:16 2022] GET /identity/v3/roles/5cd206a783e44ac2829c7661b34bb6a7 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:16.525522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e67e13-97f5-4735-b14f-1e3e0f564f62 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:16.541832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:16.626203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e67e13-97f5-4735-b14f-1e3e0f564f62 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:16.660380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1933f-99af-4415-9b28-258247dc023f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:16.660990 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1933f-99af-4415-9b28-258247dc023f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:16.661266 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1933f-99af-4415-9b28-258247dc023f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:16.661892 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1933f-99af-4415-9b28-258247dc023f None admin] PATH_INFO: `/v3/users/25f4dbf1ab364cf09c1f80b5b241a83f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:16.674678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1933f-99af-4415-9b28-258247dc023f None admin] RBAC: Authorizing `identity:get_user(user_id=25f4dbf1ab364cf09c1f80b5b241a83f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:16.676138 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1933f-99af-4415-9b28-258247dc023f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:16.677706 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 421/847] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:16 2022] GET /identity/v3/users/25f4dbf1ab364cf09c1f80b5b241a83f => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:16.686233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbd1b5-0052-4b51-9895-6965349a7d84 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:16.789257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbd1b5-0052-4b51-9895-6965349a7d84 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:16.824239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db0fc5-191b-4d67-88b6-22ac77d5153a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:16.824792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0fc5-191b-4d67-88b6-22ac77d5153a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:16.825069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0fc5-191b-4d67-88b6-22ac77d5153a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:16.825336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0fc5-191b-4d67-88b6-22ac77d5153a None admin] PATH_INFO: `/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:16.826569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0fc5-191b-4d67-88b6-22ac77d5153a None admin] RBAC: Authorizing `identity:get_project(project_id=e4a2074e9f454558bc2d3ed1e3ebd255)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:16.828025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0fc5-191b-4d67-88b6-22ac77d5153a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:16.829605 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 427/848] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:36:16 2022] GET /identity/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:36:16.836153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1933f-99af-4415-9b28-258247dc023f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:16.907876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:16.937672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1933f-99af-4415-9b28-258247dc023f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:16.971620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77cc4c8-779e-44b1-a468-485d3e49bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:16.972252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cc4c8-779e-44b1-a468-485d3e49bd3d None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:16.972523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cc4c8-779e-44b1-a468-485d3e49bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:16.972791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77cc4c8-779e-44b1-a468-485d3e49bd3d None admin] PATH_INFO: `/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255/users/25f4dbf1ab364cf09c1f80b5b241a83f/roles/5cd206a783e44ac2829c7661b34bb6a7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:16.975279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cc4c8-779e-44b1-a468-485d3e49bd3d None admin] RBAC: Authorizing `identity:create_grant(project_id=e4a2074e9f454558bc2d3ed1e3ebd255, user_id=25f4dbf1ab364cf09c1f80b5b241a83f, role_id=5cd206a783e44ac2829c7661b34bb6a7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:16.976686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77cc4c8-779e-44b1-a468-485d3e49bd3d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:16.995713 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 422/849] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:36:16 2022] PUT /identity/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255/users/25f4dbf1ab364cf09c1f80b5b241a83f/roles/5cd206a783e44ac2829c7661b34bb6a7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:36:19.098628 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dff0e-af24-48a3-8c01-6da8983b0350 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:19.099242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dff0e-af24-48a3-8c01-6da8983b0350 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:19.099242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dff0e-af24-48a3-8c01-6da8983b0350 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:19.100581 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 428/850] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:19.107196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e874cfc-4a25-44f3-9064-c8295f53baa7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:19.107953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e874cfc-4a25-44f3-9064-c8295f53baa7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:19.108071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e874cfc-4a25-44f3-9064-c8295f53baa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:19.131510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9e874cfc-4a25-44f3-9064-c8295f53baa7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:19.132695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9e874cfc-4a25-44f3-9064-c8295f53baa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:19.201051 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 423/851] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:19.466355 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dff0e-af24-48a3-8c01-6da8983b0350 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:19.533352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:19.581226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dff0e-af24-48a3-8c01-6da8983b0350 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:19.625478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba27351-ed7a-4744-855e-6dc1ab9e74f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:19.626101 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba27351-ed7a-4744-855e-6dc1ab9e74f9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:19.626379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba27351-ed7a-4744-855e-6dc1ab9e74f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:19.626646 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba27351-ed7a-4744-855e-6dc1ab9e74f9 None admin] PATH_INFO: `/v3/roles/5cd206a783e44ac2829c7661b34bb6a7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:19.628097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba27351-ed7a-4744-855e-6dc1ab9e74f9 None admin] RBAC: Authorizing `identity:get_role(role_id=5cd206a783e44ac2829c7661b34bb6a7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:19.629472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba27351-ed7a-4744-855e-6dc1ab9e74f9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:19.630789 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 429/852] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:19 2022] GET /identity/v3/roles/5cd206a783e44ac2829c7661b34bb6a7 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:19.636833 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e874cfc-4a25-44f3-9064-c8295f53baa7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:19.698737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:19.738518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e874cfc-4a25-44f3-9064-c8295f53baa7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:19.775115 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d076da9-4944-44c5-ae9d-d42642093c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:19.775797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d076da9-4944-44c5-ae9d-d42642093c12 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:19.776084 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d076da9-4944-44c5-ae9d-d42642093c12 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:19.776351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d076da9-4944-44c5-ae9d-d42642093c12 None admin] PATH_INFO: `/v3/users/25f4dbf1ab364cf09c1f80b5b241a83f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:19.777476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d076da9-4944-44c5-ae9d-d42642093c12 None admin] RBAC: Authorizing `identity:get_user(user_id=25f4dbf1ab364cf09c1f80b5b241a83f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:19.778950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d076da9-4944-44c5-ae9d-d42642093c12 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:19.780490 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 424/853] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:19 2022] GET /identity/v3/users/25f4dbf1ab364cf09c1f80b5b241a83f => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:19.790072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba27351-ed7a-4744-855e-6dc1ab9e74f9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:19.898747 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba27351-ed7a-4744-855e-6dc1ab9e74f9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:19.925193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:19.936937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffa38a-491f-4bc2-94d4-922e4138b057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:19.937542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffa38a-491f-4bc2-94d4-922e4138b057 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:19.937825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffa38a-491f-4bc2-94d4-922e4138b057 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:19.938102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffa38a-491f-4bc2-94d4-922e4138b057 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:20.014366 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffa38a-491f-4bc2-94d4-922e4138b057 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:20.016526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffa38a-491f-4bc2-94d4-922e4138b057 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:20.119662 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 430/854] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:36:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:36:20.125925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d076da9-4944-44c5-ae9d-d42642093c12 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:20.229993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d076da9-4944-44c5-ae9d-d42642093c12 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:20.257965 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:20.265743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32f098-7e9b-44f1-9bbd-509f805e6bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:20.266312 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32f098-7e9b-44f1-9bbd-509f805e6bf8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:20.266586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32f098-7e9b-44f1-9bbd-509f805e6bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:20.266884 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32f098-7e9b-44f1-9bbd-509f805e6bf8 None admin] PATH_INFO: `/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:20.268155 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32f098-7e9b-44f1-9bbd-509f805e6bf8 None admin] RBAC: Authorizing `identity:get_project(project_id=e4a2074e9f454558bc2d3ed1e3ebd255)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:20.269615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32f098-7e9b-44f1-9bbd-509f805e6bf8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:20.271286 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 425/855] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:36:20 2022] GET /identity/v3/projects/e4a2074e9f454558bc2d3ed1e3ebd255 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:20.277662 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffa38a-491f-4bc2-94d4-922e4138b057 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:20.307180 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:20.381059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffa38a-491f-4bc2-94d4-922e4138b057 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:20.418232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc87b0c-2599-4d2d-a7e1-41a468cba51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:20.418860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc87b0c-2599-4d2d-a7e1-41a468cba51e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:20.419141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc87b0c-2599-4d2d-a7e1-41a468cba51e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:20.419414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc87b0c-2599-4d2d-a7e1-41a468cba51e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:20.420131 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc87b0c-2599-4d2d-a7e1-41a468cba51e None admin] RBAC: Adding query filter params (role.id=5cd206a783e44ac2829c7661b34bb6a7, scope.project.id=e4a2074e9f454558bc2d3ed1e3ebd255, user.id=25f4dbf1ab364cf09c1f80b5b241a83f) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:20.420434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc87b0c-2599-4d2d-a7e1-41a468cba51e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:20.421863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc87b0c-2599-4d2d-a7e1-41a468cba51e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:20.427670 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 431/856] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:36:20 2022] GET /identity/v3/role_assignments?user.id=25f4dbf1ab364cf09c1f80b5b241a83f&scope.project.id=e4a2074e9f454558bc2d3ed1e3ebd255&role.id=5cd206a783e44ac2829c7661b34bb6a7 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:22.577714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb5b59-756e-4818-b230-30fdcf3db8dc None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:22.578531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb5b59-756e-4818-b230-30fdcf3db8dc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:22.578531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb5b59-756e-4818-b230-30fdcf3db8dc None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:22.580765 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 426/857] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:22.587144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ab79e-96b4-40cf-b7e8-3d177ca2a245 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:22.587824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ab79e-96b4-40cf-b7e8-3d177ca2a245 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:22.588013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ab79e-96b4-40cf-b7e8-3d177ca2a245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:22.613161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-830ab79e-96b4-40cf-b7e8-3d177ca2a245 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:22.614331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-830ab79e-96b4-40cf-b7e8-3d177ca2a245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:22.686243 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 432/858] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:22.950037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb5b59-756e-4818-b230-30fdcf3db8dc None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:23.057659 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb5b59-756e-4818-b230-30fdcf3db8dc None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:23.092022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1ef75-acbe-4038-9a89-c6d574a3ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:23.092628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1ef75-acbe-4038-9a89-c6d574a3ffda None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:23.092902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1ef75-acbe-4038-9a89-c6d574a3ffda None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:23.093168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1ef75-acbe-4038-9a89-c6d574a3ffda None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:23.094392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1ef75-acbe-4038-9a89-c6d574a3ffda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:23.095856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1ef75-acbe-4038-9a89-c6d574a3ffda None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:23.097376 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 427/859] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:23.105957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ab79e-96b4-40cf-b7e8-3d177ca2a245 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:23.131465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:23.209393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ab79e-96b4-40cf-b7e8-3d177ca2a245 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:23.244261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd8ab0-da2c-4fd1-a86a-b6ca3887431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:23.244893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd8ab0-da2c-4fd1-a86a-b6ca3887431e None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:23.245168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd8ab0-da2c-4fd1-a86a-b6ca3887431e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:23.245437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd8ab0-da2c-4fd1-a86a-b6ca3887431e None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:23.246091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd8ab0-da2c-4fd1-a86a-b6ca3887431e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:23.247541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd8ab0-da2c-4fd1-a86a-b6ca3887431e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:23.282152 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 433/860] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:36:23 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:36:25.458770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577bbab-bd7b-453c-b9d6-a5826f19c742 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:25.459325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577bbab-bd7b-453c-b9d6-a5826f19c742 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:25.459325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577bbab-bd7b-453c-b9d6-a5826f19c742 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:25.460645 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 428/861] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:25.467170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b5f0d-717d-4d6e-a406-1a132db8b2a1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:25.467888 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b5f0d-717d-4d6e-a406-1a132db8b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:25.468035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b5f0d-717d-4d6e-a406-1a132db8b2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:25.491303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-204b5f0d-717d-4d6e-a406-1a132db8b2a1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:25.492476 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-204b5f0d-717d-4d6e-a406-1a132db8b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:25.560587 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 434/862] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:25.831593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577bbab-bd7b-453c-b9d6-a5826f19c742 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:25.907083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:25.944013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577bbab-bd7b-453c-b9d6-a5826f19c742 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:25.978394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3f561-cc39-4ae2-914d-e308610f5959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:25.979095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3f561-cc39-4ae2-914d-e308610f5959 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:25.979344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3f561-cc39-4ae2-914d-e308610f5959 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:25.979628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3f561-cc39-4ae2-914d-e308610f5959 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:25.981594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3f561-cc39-4ae2-914d-e308610f5959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:25.984229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3f561-cc39-4ae2-914d-e308610f5959 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:25.986639 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 429/863] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:36:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:25.993184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b5f0d-717d-4d6e-a406-1a132db8b2a1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:25.997048 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:26.097034 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b5f0d-717d-4d6e-a406-1a132db8b2a1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:26.132865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab510f29-ba47-40df-b538-3272e1d3796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:26.133463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab510f29-ba47-40df-b538-3272e1d3796f None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:26.133741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab510f29-ba47-40df-b538-3272e1d3796f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:26.134015 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab510f29-ba47-40df-b538-3272e1d3796f None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:26.134656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab510f29-ba47-40df-b538-3272e1d3796f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:26.136227 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab510f29-ba47-40df-b538-3272e1d3796f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:26.159047 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 435/864] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:36:25 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:36:28.295805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45501256-e4ed-4c26-b722-e85fd1e99d83 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:28.296273 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45501256-e4ed-4c26-b722-e85fd1e99d83 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:28.296384 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45501256-e4ed-4c26-b722-e85fd1e99d83 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:28.298329 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 430/865] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:28.306228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d4a29-88ef-47da-a990-469d9a48e9cd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:28.306638 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d4a29-88ef-47da-a990-469d9a48e9cd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:28.306825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d4a29-88ef-47da-a990-469d9a48e9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:28.325922 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cb2d4a29-88ef-47da-a990-469d9a48e9cd None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:28.327144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cb2d4a29-88ef-47da-a990-469d9a48e9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:28.385175 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:28.399119 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 436/866] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:28.663658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45501256-e4ed-4c26-b722-e85fd1e99d83 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:28.770857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45501256-e4ed-4c26-b722-e85fd1e99d83 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:28.805579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97098989-0e46-48af-971a-15bc3cc167df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:28.806173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97098989-0e46-48af-971a-15bc3cc167df None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:28.806442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97098989-0e46-48af-971a-15bc3cc167df None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:28.806706 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97098989-0e46-48af-971a-15bc3cc167df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:28.810055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:28.826236 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97098989-0e46-48af-971a-15bc3cc167df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-97098989-0e46-48af-971a-15bc3cc167df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:36:28.834059 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:28.835225 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:36:28.836234 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 431/867] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:36:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:28.841800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d4a29-88ef-47da-a990-469d9a48e9cd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:28.942297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d4a29-88ef-47da-a990-469d9a48e9cd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:28.977505 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8d652-0254-4d06-80b3-4e65f96c6dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:28.978103 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8d652-0254-4d06-80b3-4e65f96c6dae None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:28.978514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8d652-0254-4d06-80b3-4e65f96c6dae None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:28.978616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8d652-0254-4d06-80b3-4e65f96c6dae None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:28.979354 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8d652-0254-4d06-80b3-4e65f96c6dae None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:28.979621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8d652-0254-4d06-80b3-4e65f96c6dae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:28.981088 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8d652-0254-4d06-80b3-4e65f96c6dae None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:28.989693 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 437/868] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:36:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:36:28.996388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97098989-0e46-48af-971a-15bc3cc167df None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:29.100126 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97098989-0e46-48af-971a-15bc3cc167df None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:29.134863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34635e40-ac12-4c27-ab2c-612c4a50cb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:29.135532 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34635e40-ac12-4c27-ab2c-612c4a50cb69 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:29.135809 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34635e40-ac12-4c27-ab2c-612c4a50cb69 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:29.136083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34635e40-ac12-4c27-ab2c-612c4a50cb69 None admin] PATH_INFO: `/v3/users/1a8dc90aecdd4f22b3b1ca5e61ee0375` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:29.150544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34635e40-ac12-4c27-ab2c-612c4a50cb69 None admin] RBAC: Authorizing `identity:get_user(user_id=1a8dc90aecdd4f22b3b1ca5e61ee0375)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:29.152889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34635e40-ac12-4c27-ab2c-612c4a50cb69 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:29.155425 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 432/869] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:28 2022] GET /identity/v3/users/1a8dc90aecdd4f22b3b1ca5e61ee0375 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:29.165031 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8d652-0254-4d06-80b3-4e65f96c6dae None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:29.220688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:29.275110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8d652-0254-4d06-80b3-4e65f96c6dae None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:29.309350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3517a1-b062-4f57-abac-189079b95ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:29.309979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3517a1-b062-4f57-abac-189079b95ea2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:29.310260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3517a1-b062-4f57-abac-189079b95ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:29.310524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3517a1-b062-4f57-abac-189079b95ea2 None admin] PATH_INFO: `/v3/projects/3163004da244439cb2dc595922aae9ee` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:29.311796 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3517a1-b062-4f57-abac-189079b95ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=3163004da244439cb2dc595922aae9ee)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:29.313233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3517a1-b062-4f57-abac-189079b95ea2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:29.314859 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 438/870] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:36:29 2022] GET /identity/v3/projects/3163004da244439cb2dc595922aae9ee => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:29.320987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34635e40-ac12-4c27-ab2c-612c4a50cb69 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:29.375145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:29.427370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34635e40-ac12-4c27-ab2c-612c4a50cb69 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:29.461378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57ad16-b93c-4ae5-afda-7b4a2a640d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:29.462011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57ad16-b93c-4ae5-afda-7b4a2a640d34 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:29.462282 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57ad16-b93c-4ae5-afda-7b4a2a640d34 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:29.462546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57ad16-b93c-4ae5-afda-7b4a2a640d34 None admin] PATH_INFO: `/v3/projects/3163004da244439cb2dc595922aae9ee/users/1a8dc90aecdd4f22b3b1ca5e61ee0375/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:29.464711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57ad16-b93c-4ae5-afda-7b4a2a640d34 None admin] RBAC: Authorizing `identity:create_grant(project_id=3163004da244439cb2dc595922aae9ee, user_id=1a8dc90aecdd4f22b3b1ca5e61ee0375, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:29.466131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57ad16-b93c-4ae5-afda-7b4a2a640d34 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:29.483152 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 433/871] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:36:29 2022] PUT /identity/v3/projects/3163004da244439cb2dc595922aae9ee/users/1a8dc90aecdd4f22b3b1ca5e61ee0375/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:36:31.589312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a891fe-d240-4bcf-a238-fed1ef0d4165 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:31.589826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a891fe-d240-4bcf-a238-fed1ef0d4165 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:31.589826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a891fe-d240-4bcf-a238-fed1ef0d4165 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:31.591917 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 439/872] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:31.599711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ced634-35ee-4b71-a9e7-cf809ff2fb79 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:31.600147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ced634-35ee-4b71-a9e7-cf809ff2fb79 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:31.600255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ced634-35ee-4b71-a9e7-cf809ff2fb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:31.619073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-48ced634-35ee-4b71-a9e7-cf809ff2fb79 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:31.620369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-48ced634-35ee-4b71-a9e7-cf809ff2fb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:31.688690 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 434/873] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:36:31.950260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a891fe-d240-4bcf-a238-fed1ef0d4165 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:32.069336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a891fe-d240-4bcf-a238-fed1ef0d4165 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:32.089172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:32.105479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e455aeb-2820-40d8-bb2e-825ef9f0f988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:32.106121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e455aeb-2820-40d8-bb2e-825ef9f0f988 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:32.106400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e455aeb-2820-40d8-bb2e-825ef9f0f988 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:32.106677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e455aeb-2820-40d8-bb2e-825ef9f0f988 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:32.118419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e455aeb-2820-40d8-bb2e-825ef9f0f988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-9e455aeb-2820-40d8-bb2e-825ef9f0f988 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:36:32.123290 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:32.124423 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:32.125273 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 440/874] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:36:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:32.130582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ced634-35ee-4b71-a9e7-cf809ff2fb79 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:32.175006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:32.238810 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ced634-35ee-4b71-a9e7-cf809ff2fb79 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:32.279426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89a8d7-2891-4dc3-b6cb-ecbf59f755c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:32.280022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89a8d7-2891-4dc3-b6cb-ecbf59f755c5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:32.280291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89a8d7-2891-4dc3-b6cb-ecbf59f755c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:32.280554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89a8d7-2891-4dc3-b6cb-ecbf59f755c5 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:32.281210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89a8d7-2891-4dc3-b6cb-ecbf59f755c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:32.281504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89a8d7-2891-4dc3-b6cb-ecbf59f755c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:32.282895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89a8d7-2891-4dc3-b6cb-ecbf59f755c5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:32.291687 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 435/875] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:36:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:32.297850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e455aeb-2820-40d8-bb2e-825ef9f0f988 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:32.407881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e455aeb-2820-40d8-bb2e-825ef9f0f988 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:32.442050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4015455-43cf-49b4-852c-6cc59e026c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:32.442676 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4015455-43cf-49b4-852c-6cc59e026c84 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:32.442987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4015455-43cf-49b4-852c-6cc59e026c84 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:32.443258 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4015455-43cf-49b4-852c-6cc59e026c84 None admin] PATH_INFO: `/v3/users/1a8dc90aecdd4f22b3b1ca5e61ee0375` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:32.444413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4015455-43cf-49b4-852c-6cc59e026c84 None admin] RBAC: Authorizing `identity:get_user(user_id=1a8dc90aecdd4f22b3b1ca5e61ee0375)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:32.445838 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4015455-43cf-49b4-852c-6cc59e026c84 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:32.447530 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 441/876] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:32 2022] GET /identity/v3/users/1a8dc90aecdd4f22b3b1ca5e61ee0375 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:32.453781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89a8d7-2891-4dc3-b6cb-ecbf59f755c5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:32.560901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89a8d7-2891-4dc3-b6cb-ecbf59f755c5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:32.563169 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:32.604213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c069838-a0be-4f12-b5f4-06af4b6e6afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:32.604813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c069838-a0be-4f12-b5f4-06af4b6e6afc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:32.605070 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c069838-a0be-4f12-b5f4-06af4b6e6afc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:32.605348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c069838-a0be-4f12-b5f4-06af4b6e6afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:32.672932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c069838-a0be-4f12-b5f4-06af4b6e6afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:32.674358 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c069838-a0be-4f12-b5f4-06af4b6e6afc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:32.774799 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 436/877] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:36:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:36:32.780926 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4015455-43cf-49b4-852c-6cc59e026c84 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:32.832188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:32.885090 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4015455-43cf-49b4-852c-6cc59e026c84 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:32.920013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc02b01-f647-41e0-af8d-557004bbd73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:32.920620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc02b01-f647-41e0-af8d-557004bbd73e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:32.920894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc02b01-f647-41e0-af8d-557004bbd73e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:32.921170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc02b01-f647-41e0-af8d-557004bbd73e None admin] PATH_INFO: `/v3/projects/3163004da244439cb2dc595922aae9ee` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:32.922302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc02b01-f647-41e0-af8d-557004bbd73e None admin] RBAC: Authorizing `identity:get_project(project_id=3163004da244439cb2dc595922aae9ee)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:32.923745 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc02b01-f647-41e0-af8d-557004bbd73e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:32.925334 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 442/878] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:36:32 2022] GET /identity/v3/projects/3163004da244439cb2dc595922aae9ee => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:32.931774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c069838-a0be-4f12-b5f4-06af4b6e6afc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:32.935200 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:33.036356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c069838-a0be-4f12-b5f4-06af4b6e6afc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:33.076939 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7956c7e4-ec2f-4cc4-be97-baa225f799e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:33.077524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956c7e4-ec2f-4cc4-be97-baa225f799e4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:33.077799 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956c7e4-ec2f-4cc4-be97-baa225f799e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:33.078069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7956c7e4-ec2f-4cc4-be97-baa225f799e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:33.078820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956c7e4-ec2f-4cc4-be97-baa225f799e4 None admin] RBAC: Adding query filter params (role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.project.id=3163004da244439cb2dc595922aae9ee, user.id=1a8dc90aecdd4f22b3b1ca5e61ee0375) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:33.079118 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956c7e4-ec2f-4cc4-be97-baa225f799e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:33.080542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7956c7e4-ec2f-4cc4-be97-baa225f799e4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:33.086428 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 437/879] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:36:32 2022] GET /identity/v3/role_assignments?user.id=1a8dc90aecdd4f22b3b1ca5e61ee0375&scope.project.id=3163004da244439cb2dc595922aae9ee&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:36:35.198335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1ba90-aac7-4da6-8b5e-99cbcecbff8b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:35.199407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1ba90-aac7-4da6-8b5e-99cbcecbff8b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:35.199407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1ba90-aac7-4da6-8b5e-99cbcecbff8b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:35.201533 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 443/880] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:35.208042 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9866a-b9f3-48db-a7f4-e5b2ee16b1c0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:35.208765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9866a-b9f3-48db-a7f4-e5b2ee16b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:35.208914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9866a-b9f3-48db-a7f4-e5b2ee16b1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:35.235357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a9d9866a-b9f3-48db-a7f4-e5b2ee16b1c0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:35.236541 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a9d9866a-b9f3-48db-a7f4-e5b2ee16b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:35.309151 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:35.315430 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 438/881] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:35.581415 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1ba90-aac7-4da6-8b5e-99cbcecbff8b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:35.692515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1ba90-aac7-4da6-8b5e-99cbcecbff8b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:35.708167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:35.727029 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff235d2-7706-489b-8c01-355f797225c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:35.727663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff235d2-7706-489b-8c01-355f797225c7 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:35.727946 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff235d2-7706-489b-8c01-355f797225c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:35.728221 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff235d2-7706-489b-8c01-355f797225c7 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:35.728910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff235d2-7706-489b-8c01-355f797225c7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:35.730265 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff235d2-7706-489b-8c01-355f797225c7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:35.747317 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 444/882] 10.222.0.26 () {60 vars in 1138 bytes} [Mon Oct 24 08:36:35 2022] POST /identity/v3/domains => generated 267 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:36:37.923521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d876af7-44fb-45f1-8f41-78c84cf8eb95 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:37.924428 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d876af7-44fb-45f1-8f41-78c84cf8eb95 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:37.924428 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d876af7-44fb-45f1-8f41-78c84cf8eb95 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:37.926509 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 439/883] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:37.932193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744df0c-d585-4fc3-be95-99cb572cf7f5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:37.932625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744df0c-d585-4fc3-be95-99cb572cf7f5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:37.932725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744df0c-d585-4fc3-be95-99cb572cf7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:37.952711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4744df0c-d585-4fc3-be95-99cb572cf7f5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:37.953887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4744df0c-d585-4fc3-be95-99cb572cf7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:38.024446 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 445/884] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:38.301921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d876af7-44fb-45f1-8f41-78c84cf8eb95 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:38.405374 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d876af7-44fb-45f1-8f41-78c84cf8eb95 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:38.439399 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c50e9-8861-4edd-9b30-e2878380d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:38.439995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c50e9-8861-4edd-9b30-e2878380d715 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:38.440265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c50e9-8861-4edd-9b30-e2878380d715 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:38.440535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c50e9-8861-4edd-9b30-e2878380d715 None admin] PATH_INFO: `/v3/domains/e6190ac5d1f24aa8b15a8483ccc80c91` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:38.450876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c50e9-8861-4edd-9b30-e2878380d715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e6190ac5d1f24aa8b15a8483ccc80c91)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:38.452305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c50e9-8861-4edd-9b30-e2878380d715 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:38.453845 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 440/885] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Oct 24 08:36:38 2022] GET /identity/v3/domains/e6190ac5d1f24aa8b15a8483ccc80c91 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:38.459793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744df0c-d585-4fc3-be95-99cb572cf7f5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:38.684725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744df0c-d585-4fc3-be95-99cb572cf7f5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:38.691526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:38.728176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1c607-4daa-412a-b671-e41824e89436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:38.728776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1c607-4daa-412a-b671-e41824e89436 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:38.729056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1c607-4daa-412a-b671-e41824e89436 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:38.729320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1c607-4daa-412a-b671-e41824e89436 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:38.729982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1c607-4daa-412a-b671-e41824e89436 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:38.731438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1c607-4daa-412a-b671-e41824e89436 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:38.764722 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 446/886] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:36:38 2022] POST /identity/v3/projects => generated 361 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:36:40.925265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420074c9-f70b-4871-9942-b812a7abc8f8 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:40.926173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420074c9-f70b-4871-9942-b812a7abc8f8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:40.926173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420074c9-f70b-4871-9942-b812a7abc8f8 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:40.928290 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 441/887] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:40.936178 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad370231-39dd-4d02-a6d9-6e68dc0d809e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:40.936594 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad370231-39dd-4d02-a6d9-6e68dc0d809e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:40.936791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad370231-39dd-4d02-a6d9-6e68dc0d809e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:40.964675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ad370231-39dd-4d02-a6d9-6e68dc0d809e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:40.965872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ad370231-39dd-4d02-a6d9-6e68dc0d809e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:41.049546 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 447/888] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:41.316718 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420074c9-f70b-4871-9942-b812a7abc8f8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:41.383983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:41.427795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420074c9-f70b-4871-9942-b812a7abc8f8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:41.463742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec6923-daf2-4240-b1e3-479bb1482bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:41.464354 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6923-daf2-4240-b1e3-479bb1482bc1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:41.464624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6923-daf2-4240-b1e3-479bb1482bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:41.464891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6923-daf2-4240-b1e3-479bb1482bc1 None admin] PATH_INFO: `/v3/domains/e6190ac5d1f24aa8b15a8483ccc80c91` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:41.466045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec6923-daf2-4240-b1e3-479bb1482bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e6190ac5d1f24aa8b15a8483ccc80c91)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:41.467495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec6923-daf2-4240-b1e3-479bb1482bc1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:41.469018 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 442/889] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Oct 24 08:36:41 2022] GET /identity/v3/domains/e6190ac5d1f24aa8b15a8483ccc80c91 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:41.475191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad370231-39dd-4d02-a6d9-6e68dc0d809e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:41.564713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:41.578041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad370231-39dd-4d02-a6d9-6e68dc0d809e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:41.611949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141eeb6a-0721-4544-ba93-c9115d6dff67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:41.612541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141eeb6a-0721-4544-ba93-c9115d6dff67 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:41.612812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141eeb6a-0721-4544-ba93-c9115d6dff67 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:41.613081 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141eeb6a-0721-4544-ba93-c9115d6dff67 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:41.613723 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141eeb6a-0721-4544-ba93-c9115d6dff67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:41.615185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141eeb6a-0721-4544-ba93-c9115d6dff67 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:41.638497 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 448/890] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:36:41 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:36:43.794753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78389a09-a8ee-4fd4-902b-82dd34070299 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:43.795375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78389a09-a8ee-4fd4-902b-82dd34070299 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:43.795375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78389a09-a8ee-4fd4-902b-82dd34070299 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:43.796656 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 443/891] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:43.803169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed068fc-8d52-42f4-a6ab-b07c149de7b6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:43.803946 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed068fc-8d52-42f4-a6ab-b07c149de7b6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:43.804092 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed068fc-8d52-42f4-a6ab-b07c149de7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:43.825129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0ed068fc-8d52-42f4-a6ab-b07c149de7b6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:43.826314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0ed068fc-8d52-42f4-a6ab-b07c149de7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:43.895335 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 449/892] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:44.158982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78389a09-a8ee-4fd4-902b-82dd34070299 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:44.267724 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78389a09-a8ee-4fd4-902b-82dd34070299 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:44.296034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:44.306753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bfd97-3034-4ca2-b1b3-6a800d9b887e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:44.307406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bfd97-3034-4ca2-b1b3-6a800d9b887e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:44.307685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bfd97-3034-4ca2-b1b3-6a800d9b887e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:44.307954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bfd97-3034-4ca2-b1b3-6a800d9b887e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:44.319002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bfd97-3034-4ca2-b1b3-6a800d9b887e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-380bfd97-3034-4ca2-b1b3-6a800d9b887e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:36:44.323794 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:44.324865 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:36:44.325830 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 444/893] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:36:44.333070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed068fc-8d52-42f4-a6ab-b07c149de7b6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:44.408201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:44.436408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed068fc-8d52-42f4-a6ab-b07c149de7b6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:44.470390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a86b2-926e-444b-b33b-5b8dab405862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:44.471008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a86b2-926e-444b-b33b-5b8dab405862 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:44.471282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a86b2-926e-444b-b33b-5b8dab405862 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:44.471550 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a86b2-926e-444b-b33b-5b8dab405862 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:44.472207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a86b2-926e-444b-b33b-5b8dab405862 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:44.472506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a86b2-926e-444b-b33b-5b8dab405862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:44.473864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a86b2-926e-444b-b33b-5b8dab405862 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:44.481660 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 450/894] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:36:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:44.487842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bfd97-3034-4ca2-b1b3-6a800d9b887e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:44.602620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bfd97-3034-4ca2-b1b3-6a800d9b887e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:44.636552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d7d37-3a94-4acf-b09d-3efde7c27a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:44.637174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d7d37-3a94-4acf-b09d-3efde7c27a72 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:44.637447 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d7d37-3a94-4acf-b09d-3efde7c27a72 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:44.637716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d7d37-3a94-4acf-b09d-3efde7c27a72 None admin] PATH_INFO: `/v3/users/b40722d4ce5d4a05b25ec4901d13ed8a` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:44.652021 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d7d37-3a94-4acf-b09d-3efde7c27a72 None admin] RBAC: Authorizing `identity:get_user(user_id=b40722d4ce5d4a05b25ec4901d13ed8a)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:44.653445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d7d37-3a94-4acf-b09d-3efde7c27a72 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:44.655044 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 445/895] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:44 2022] GET /identity/v3/users/b40722d4ce5d4a05b25ec4901d13ed8a => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:44.661001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a86b2-926e-444b-b33b-5b8dab405862 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:44.761971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a86b2-926e-444b-b33b-5b8dab405862 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:44.795805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a2e90-63c4-49ca-8813-8d80d8a69f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:44.796419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a2e90-63c4-49ca-8813-8d80d8a69f20 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:44.796692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a2e90-63c4-49ca-8813-8d80d8a69f20 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:44.796938 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a2e90-63c4-49ca-8813-8d80d8a69f20 None admin] PATH_INFO: `/v3/projects/e40bcc6474d8409ca8a86602ec80deed` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:44.798054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a2e90-63c4-49ca-8813-8d80d8a69f20 None admin] RBAC: Authorizing `identity:get_project(project_id=e40bcc6474d8409ca8a86602ec80deed)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:44.799491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a2e90-63c4-49ca-8813-8d80d8a69f20 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:44.801070 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 451/896] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:36:44 2022] GET /identity/v3/projects/e40bcc6474d8409ca8a86602ec80deed => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:44.809531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d7d37-3a94-4acf-b09d-3efde7c27a72 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:44.860663 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:44.920765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d7d37-3a94-4acf-b09d-3efde7c27a72 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:44.955003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d301f-4b48-4256-b853-ab6f4d036199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:44.955645 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d301f-4b48-4256-b853-ab6f4d036199 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:44.955919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d301f-4b48-4256-b853-ab6f4d036199 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:44.956188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d301f-4b48-4256-b853-ab6f4d036199 None admin] PATH_INFO: `/v3/projects/e40bcc6474d8409ca8a86602ec80deed/users/b40722d4ce5d4a05b25ec4901d13ed8a/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:44.958310 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d301f-4b48-4256-b853-ab6f4d036199 None admin] RBAC: Authorizing `identity:create_grant(project_id=e40bcc6474d8409ca8a86602ec80deed, user_id=b40722d4ce5d4a05b25ec4901d13ed8a, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:44.959766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d301f-4b48-4256-b853-ab6f4d036199 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:44.976167 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 446/897] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:36:44 2022] PUT /identity/v3/projects/e40bcc6474d8409ca8a86602ec80deed/users/b40722d4ce5d4a05b25ec4901d13ed8a/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:36:47.072129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0a8ad-96dd-432b-8874-ad6f62ace30d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:47.072992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0a8ad-96dd-432b-8874-ad6f62ace30d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:47.072992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0a8ad-96dd-432b-8874-ad6f62ace30d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:47.075275 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 452/898] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:36:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:36:47.080696 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0b785-06ce-4e30-b180-3be12b25802b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:47.081139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0b785-06ce-4e30-b180-3be12b25802b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:47.081235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0b785-06ce-4e30-b180-3be12b25802b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:47.100156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-35f0b785-06ce-4e30-b180-3be12b25802b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:36:47.101341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-35f0b785-06ce-4e30-b180-3be12b25802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:36:47.172372 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 447/899] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:36:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:36:47.434731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0a8ad-96dd-432b-8874-ad6f62ace30d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:47.438868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:47.547850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0a8ad-96dd-432b-8874-ad6f62ace30d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:47.586250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8ebee-16b7-48fc-82f7-2f441c231d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:47.586898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8ebee-16b7-48fc-82f7-2f441c231d3b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:47.587176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8ebee-16b7-48fc-82f7-2f441c231d3b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:47.587444 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8ebee-16b7-48fc-82f7-2f441c231d3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:47.598121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8ebee-16b7-48fc-82f7-2f441c231d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-6af8ebee-16b7-48fc-82f7-2f441c231d3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:36:47.602818 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:36:47.603869 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:36:47.604819 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 453/900] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:36:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:36:47.610085 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0b785-06ce-4e30-b180-3be12b25802b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:47.648716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:47.716057 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0b785-06ce-4e30-b180-3be12b25802b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:47.753168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987667f-7430-4b3c-8bf0-c5695133ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:47.753767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987667f-7430-4b3c-8bf0-c5695133ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:47.754040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987667f-7430-4b3c-8bf0-c5695133ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:47.754305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987667f-7430-4b3c-8bf0-c5695133ecb4 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:47.754990 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987667f-7430-4b3c-8bf0-c5695133ecb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:47.755282 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987667f-7430-4b3c-8bf0-c5695133ecb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:47.756648 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6987667f-7430-4b3c-8bf0-c5695133ecb4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:47.764607 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 448/901] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:36:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:47.770680 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8ebee-16b7-48fc-82f7-2f441c231d3b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:47.834107 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:47.871850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8ebee-16b7-48fc-82f7-2f441c231d3b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:47.906638 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429f9df-8ab1-4cf6-9e67-9c243c5003bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:47.907276 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429f9df-8ab1-4cf6-9e67-9c243c5003bb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:47.907550 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429f9df-8ab1-4cf6-9e67-9c243c5003bb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:47.907816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429f9df-8ab1-4cf6-9e67-9c243c5003bb None admin] PATH_INFO: `/v3/users/b40722d4ce5d4a05b25ec4901d13ed8a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:47.908965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429f9df-8ab1-4cf6-9e67-9c243c5003bb None admin] RBAC: Authorizing `identity:get_user(user_id=b40722d4ce5d4a05b25ec4901d13ed8a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:47.910383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429f9df-8ab1-4cf6-9e67-9c243c5003bb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:47.912312 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 454/902] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Oct 24 08:36:47 2022] GET /identity/v3/users/b40722d4ce5d4a05b25ec4901d13ed8a => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:47.918229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987667f-7430-4b3c-8bf0-c5695133ecb4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:48.022764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:48.030357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987667f-7430-4b3c-8bf0-c5695133ecb4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:48.067219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6734a-c0ec-46fd-b14e-9f29e7112b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:48.067820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6734a-c0ec-46fd-b14e-9f29e7112b77 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:48.068094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6734a-c0ec-46fd-b14e-9f29e7112b77 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:48.068365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6734a-c0ec-46fd-b14e-9f29e7112b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:48.136700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6734a-c0ec-46fd-b14e-9f29e7112b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:48.138127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6734a-c0ec-46fd-b14e-9f29e7112b77 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:48.232071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:48.242836 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 449/903] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:36:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:36:48.252073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429f9df-8ab1-4cf6-9e67-9c243c5003bb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:48.353336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429f9df-8ab1-4cf6-9e67-9c243c5003bb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:48.382852 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:36:48.388813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26219228-0f81-41dc-93f0-21eac89338fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:48.389436 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26219228-0f81-41dc-93f0-21eac89338fd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:48.389711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26219228-0f81-41dc-93f0-21eac89338fd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:48.389981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26219228-0f81-41dc-93f0-21eac89338fd None admin] PATH_INFO: `/v3/projects/e40bcc6474d8409ca8a86602ec80deed` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:48.391126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26219228-0f81-41dc-93f0-21eac89338fd None admin] RBAC: Authorizing `identity:get_project(project_id=e40bcc6474d8409ca8a86602ec80deed)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:48.392554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26219228-0f81-41dc-93f0-21eac89338fd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:48.394200 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 455/904] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Oct 24 08:36:48 2022] GET /identity/v3/projects/e40bcc6474d8409ca8a86602ec80deed => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:36:48.402957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6734a-c0ec-46fd-b14e-9f29e7112b77 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:36:48.505047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6734a-c0ec-46fd-b14e-9f29e7112b77 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:36:48.539907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f894cac-6e1d-4130-a5d1-cdf6f90bbf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:36:48.540424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f894cac-6e1d-4130-a5d1-cdf6f90bbf85 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:36:48.540671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f894cac-6e1d-4130-a5d1-cdf6f90bbf85 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:36:48.540990 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f894cac-6e1d-4130-a5d1-cdf6f90bbf85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:36:48.541705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f894cac-6e1d-4130-a5d1-cdf6f90bbf85 None admin] RBAC: Adding query filter params (role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.project.id=e40bcc6474d8409ca8a86602ec80deed, user.id=b40722d4ce5d4a05b25ec4901d13ed8a) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:36:48.542029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f894cac-6e1d-4130-a5d1-cdf6f90bbf85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:36:48.543485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f894cac-6e1d-4130-a5d1-cdf6f90bbf85 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:36:48.549350 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 450/905] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:36:48 2022] GET /identity/v3/role_assignments?user.id=b40722d4ce5d4a05b25ec4901d13ed8a&scope.project.id=e40bcc6474d8409ca8a86602ec80deed&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:36:58.546804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=70 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:43.551287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae20e5-9426-431c-9aed-921e81093693 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:43.552032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae20e5-9426-431c-9aed-921e81093693 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:43.552032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae20e5-9426-431c-9aed-921e81093693 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:43.553080 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 456/906] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:37:43.558573 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483275dd-31c1-4d61-978c-fc62213a7a5f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:43.559172 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483275dd-31c1-4d61-978c-fc62213a7a5f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:43.559172 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483275dd-31c1-4d61-978c-fc62213a7a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:43.579094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-483275dd-31c1-4d61-978c-fc62213a7a5f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:37:43.580471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-483275dd-31c1-4d61-978c-fc62213a7a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:37:43.669935 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 451/907] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:37:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:37:43.935371 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae20e5-9426-431c-9aed-921e81093693 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:44.053331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae20e5-9426-431c-9aed-921e81093693 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:44.097010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8c4af-cb59-4e47-961f-936db303d27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:44.097754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8c4af-cb59-4e47-961f-936db303d27f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:44.098152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8c4af-cb59-4e47-961f-936db303d27f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:44.098348 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8c4af-cb59-4e47-961f-936db303d27f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:44.110826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8c4af-cb59-4e47-961f-936db303d27f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:44.112476 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8c4af-cb59-4e47-961f-936db303d27f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-06b8c4af-cb59-4e47-961f-936db303d27f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:37:44.134742 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:44.135934 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:37:44.136927 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:37:44.136927 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:44.136927 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:37:44.136927 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 457/908] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:44.143439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483275dd-31c1-4d61-978c-fc62213a7a5f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:44.275685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483275dd-31c1-4d61-978c-fc62213a7a5f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:44.309406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:44.316571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e163afe-3ff5-4d75-990e-625c33e393eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:44.317270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e163afe-3ff5-4d75-990e-625c33e393eb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:44.317553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e163afe-3ff5-4d75-990e-625c33e393eb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:44.317858 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e163afe-3ff5-4d75-990e-625c33e393eb None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:44.318529 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e163afe-3ff5-4d75-990e-625c33e393eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:44.318856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e163afe-3ff5-4d75-990e-625c33e393eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:44.320253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e163afe-3ff5-4d75-990e-625c33e393eb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:44.340347 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 452/909] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:44.340551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8c4af-cb59-4e47-961f-936db303d27f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:44.403930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:44.459209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8c4af-cb59-4e47-961f-936db303d27f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:44.499450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdb89e-bfc2-4032-9b9e-19cf5f57d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:44.500005 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb89e-bfc2-4032-9b9e-19cf5f57d19b None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:44.500336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb89e-bfc2-4032-9b9e-19cf5f57d19b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:44.500535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb89e-bfc2-4032-9b9e-19cf5f57d19b None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:44.501182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdb89e-bfc2-4032-9b9e-19cf5f57d19b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:44.502624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdb89e-bfc2-4032-9b9e-19cf5f57d19b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:44.534577 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 458/910] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Oct 24 08:37:44 2022] POST /identity/v3/users => generated 251 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:37:46.716697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4d1e4-0d7a-412e-82d0-62466953bbde None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:46.717362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4d1e4-0d7a-412e-82d0-62466953bbde None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:46.717362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4d1e4-0d7a-412e-82d0-62466953bbde None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:46.718853 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 453/911] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:37:46.724236 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a4aa-dd37-45c9-907b-c6127bfb5398 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:46.724713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a4aa-dd37-45c9-907b-c6127bfb5398 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:46.724829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98a4aa-dd37-45c9-907b-c6127bfb5398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:46.743700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9e98a4aa-dd37-45c9-907b-c6127bfb5398 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:37:46.744941 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9e98a4aa-dd37-45c9-907b-c6127bfb5398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:37:46.817372 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 459/912] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:37:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:37:47.081923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4d1e4-0d7a-412e-82d0-62466953bbde None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:47.200031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4d1e4-0d7a-412e-82d0-62466953bbde None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:47.238055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28c0ff-4c77-4d49-be17-10c0add813d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:47.238690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28c0ff-4c77-4d49-be17-10c0add813d4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:47.238995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28c0ff-4c77-4d49-be17-10c0add813d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:47.239263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28c0ff-4c77-4d49-be17-10c0add813d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:47.250404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc28c0ff-4c77-4d49-be17-10c0add813d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-fc28c0ff-4c77-4d49-be17-10c0add813d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:37:47.255170 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:47.256432 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:37:47.257632 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 454/913] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:47.262704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98a4aa-dd37-45c9-907b-c6127bfb5398 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:47.305032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:47.374182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98a4aa-dd37-45c9-907b-c6127bfb5398 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:47.419592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45141cdf-8834-467f-ba75-2da574690dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:47.420264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45141cdf-8834-467f-ba75-2da574690dcd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:47.420549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45141cdf-8834-467f-ba75-2da574690dcd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:47.420820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45141cdf-8834-467f-ba75-2da574690dcd None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:47.421505 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45141cdf-8834-467f-ba75-2da574690dcd None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:47.421810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45141cdf-8834-467f-ba75-2da574690dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:47.423242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45141cdf-8834-467f-ba75-2da574690dcd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:47.437882 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 460/914] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:47.442384 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28c0ff-4c77-4d49-be17-10c0add813d4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:47.509793 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:47.578650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28c0ff-4c77-4d49-be17-10c0add813d4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:47.616075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387669e9-6098-461a-84fe-a01430badc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:47.616976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387669e9-6098-461a-84fe-a01430badc6f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:47.617343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387669e9-6098-461a-84fe-a01430badc6f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:47.617711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387669e9-6098-461a-84fe-a01430badc6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:47.632435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387669e9-6098-461a-84fe-a01430badc6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:47.634211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387669e9-6098-461a-84fe-a01430badc6f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-387669e9-6098-461a-84fe-a01430badc6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:37:47.650614 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:47.652016 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:37:47.653133 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:37:47.653133 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:47.653133 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:37:47.654759 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 455/915] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 08:37:47.660054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45141cdf-8834-467f-ba75-2da574690dcd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:47.766010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:47.769070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45141cdf-8834-467f-ba75-2da574690dcd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:47.808238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be8cc2-b810-4f7a-932c-34ad1ef4eac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:47.808858 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be8cc2-b810-4f7a-932c-34ad1ef4eac5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:47.809145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be8cc2-b810-4f7a-932c-34ad1ef4eac5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:47.809401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be8cc2-b810-4f7a-932c-34ad1ef4eac5 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:47.810037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be8cc2-b810-4f7a-932c-34ad1ef4eac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:47.810336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be8cc2-b810-4f7a-932c-34ad1ef4eac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:47.811746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be8cc2-b810-4f7a-932c-34ad1ef4eac5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:47.829910 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 461/916] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:47.831856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387669e9-6098-461a-84fe-a01430badc6f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:47.937012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387669e9-6098-461a-84fe-a01430badc6f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:47.938620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:47.972552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ebc54-e74c-414c-8ff0-fb545341951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:47.973249 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ebc54-e74c-414c-8ff0-fb545341951e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:47.974591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ebc54-e74c-414c-8ff0-fb545341951e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:47.974591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ebc54-e74c-414c-8ff0-fb545341951e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:47.982908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ebc54-e74c-414c-8ff0-fb545341951e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:47.984702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ebc54-e74c-414c-8ff0-fb545341951e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0b8ebc54-e74c-414c-8ff0-fb545341951e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:37:47.998426 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:47.999830 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:37:48.005943 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 456/917] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:48.007591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be8cc2-b810-4f7a-932c-34ad1ef4eac5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:48.122149 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be8cc2-b810-4f7a-932c-34ad1ef4eac5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:48.159224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f4135-0eab-4741-92e0-f4c2ce00f508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:48.159976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f4135-0eab-4741-92e0-f4c2ce00f508 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:48.160368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f4135-0eab-4741-92e0-f4c2ce00f508 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:48.160735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f4135-0eab-4741-92e0-f4c2ce00f508 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:48.161624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f4135-0eab-4741-92e0-f4c2ce00f508 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:48.162151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f4135-0eab-4741-92e0-f4c2ce00f508 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:48.164251 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f4135-0eab-4741-92e0-f4c2ce00f508 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:48.183861 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 462/918] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Oct 24 08:37:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:48.189764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ebc54-e74c-414c-8ff0-fb545341951e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:48.321557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ebc54-e74c-414c-8ff0-fb545341951e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:48.367071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515047c-678a-43d6-80eb-df37a0179e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:48.367830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515047c-678a-43d6-80eb-df37a0179e2c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:48.368113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515047c-678a-43d6-80eb-df37a0179e2c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:48.368410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515047c-678a-43d6-80eb-df37a0179e2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:48.381078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515047c-678a-43d6-80eb-df37a0179e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:48.382752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515047c-678a-43d6-80eb-df37a0179e2c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f515047c-678a-43d6-80eb-df37a0179e2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:37:48.398748 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:48.399839 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:37:48.400845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:37:48.400845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:48.400845 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:37:48.400845 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 457/919] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:48.406212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f4135-0eab-4741-92e0-f4c2ce00f508 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:48.438592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:48.530901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f4135-0eab-4741-92e0-f4c2ce00f508 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:48.570126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2c4ce-71c8-4bad-8947-17152ef56295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:48.570880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2c4ce-71c8-4bad-8947-17152ef56295 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:48.571171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2c4ce-71c8-4bad-8947-17152ef56295 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:48.571438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2c4ce-71c8-4bad-8947-17152ef56295 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:48.572135 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2c4ce-71c8-4bad-8947-17152ef56295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:48.572471 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2c4ce-71c8-4bad-8947-17152ef56295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:48.573998 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2c4ce-71c8-4bad-8947-17152ef56295 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:48.587957 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 463/920] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:48.594296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515047c-678a-43d6-80eb-df37a0179e2c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:48.613396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:48.701482 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515047c-678a-43d6-80eb-df37a0179e2c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:48.737989 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7860126-b3bf-4c17-b1f1-21a585e76dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:48.738644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7860126-b3bf-4c17-b1f1-21a585e76dcf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:48.738908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7860126-b3bf-4c17-b1f1-21a585e76dcf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:48.739179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7860126-b3bf-4c17-b1f1-21a585e76dcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:48.746519 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7860126-b3bf-4c17-b1f1-21a585e76dcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:48.748116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7860126-b3bf-4c17-b1f1-21a585e76dcf None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f7860126-b3bf-4c17-b1f1-21a585e76dcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:37:48.759915 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:48.761951 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:48.763078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:37:48.763078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:37:48.763078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:37:48.763078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:37:48.763078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:37:48.763078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:37:48.763078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:37:48.763078 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:37:48.766199 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 458/921] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:37:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:48.767650 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2c4ce-71c8-4bad-8947-17152ef56295 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:48.853079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:48.877217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2c4ce-71c8-4bad-8947-17152ef56295 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:48.911239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067da19a-3e57-4383-8cf5-d49d15b859d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:48.911842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067da19a-3e57-4383-8cf5-d49d15b859d6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:48.912112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067da19a-3e57-4383-8cf5-d49d15b859d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:48.912391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067da19a-3e57-4383-8cf5-d49d15b859d6 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:48.913428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067da19a-3e57-4383-8cf5-d49d15b859d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:48.913729 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067da19a-3e57-4383-8cf5-d49d15b859d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:48.915205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067da19a-3e57-4383-8cf5-d49d15b859d6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:48.927640 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 464/922] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:37:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:48.933775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7860126-b3bf-4c17-b1f1-21a585e76dcf None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:49.021881 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:49.052168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7860126-b3bf-4c17-b1f1-21a585e76dcf None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:49.088634 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913e096-0ce7-43e9-b37a-15984214a048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:49.089301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913e096-0ce7-43e9-b37a-15984214a048 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:49.089583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913e096-0ce7-43e9-b37a-15984214a048 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:49.089858 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913e096-0ce7-43e9-b37a-15984214a048 None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/c3749568726644be9ebdf4d87463ac0e/roles/cb7d27022ded4ea2afd3d5ebaefbf429` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:49.106325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913e096-0ce7-43e9-b37a-15984214a048 None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=c3749568726644be9ebdf4d87463ac0e, role_id=cb7d27022ded4ea2afd3d5ebaefbf429)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:49.107918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913e096-0ce7-43e9-b37a-15984214a048 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:49.121193 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 459/923] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:37:48 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/c3749568726644be9ebdf4d87463ac0e/roles/cb7d27022ded4ea2afd3d5ebaefbf429 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:37:51.264263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113a6ea-fb4b-4de3-b7a8-67eac7010ba9 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:51.264810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113a6ea-fb4b-4de3-b7a8-67eac7010ba9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:51.264810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113a6ea-fb4b-4de3-b7a8-67eac7010ba9 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:51.266365 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 465/924] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:37:51.271775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea24fb3-c30d-4ca3-92d0-f835d3cb858c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:51.272242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea24fb3-c30d-4ca3-92d0-f835d3cb858c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:51.272342 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea24fb3-c30d-4ca3-92d0-f835d3cb858c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:51.293371 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7ea24fb3-c30d-4ca3-92d0-f835d3cb858c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:37:51.294562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7ea24fb3-c30d-4ca3-92d0-f835d3cb858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:37:51.366563 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 460/925] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:37:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:37:51.645777 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113a6ea-fb4b-4de3-b7a8-67eac7010ba9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:51.755260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113a6ea-fb4b-4de3-b7a8-67eac7010ba9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:51.791803 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899e212-5be3-42fd-b768-5d0e8f8f460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:51.792558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899e212-5be3-42fd-b768-5d0e8f8f460c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:51.792832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899e212-5be3-42fd-b768-5d0e8f8f460c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:51.793159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899e212-5be3-42fd-b768-5d0e8f8f460c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:51.800250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:51.806816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899e212-5be3-42fd-b768-5d0e8f8f460c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e899e212-5be3-42fd-b768-5d0e8f8f460c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:37:51.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:51.811465 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:37:51.812477 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 466/926] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Oct 24 08:37:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:51.817948 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea24fb3-c30d-4ca3-92d0-f835d3cb858c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:51.876776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:51.923764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea24fb3-c30d-4ca3-92d0-f835d3cb858c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:51.958350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282acd8-1185-4da4-ac9d-9b65c996cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:51.958987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282acd8-1185-4da4-ac9d-9b65c996cd77 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:51.959264 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282acd8-1185-4da4-ac9d-9b65c996cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:51.959562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282acd8-1185-4da4-ac9d-9b65c996cd77 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:51.960305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282acd8-1185-4da4-ac9d-9b65c996cd77 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:51.960641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282acd8-1185-4da4-ac9d-9b65c996cd77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:51.962047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282acd8-1185-4da4-ac9d-9b65c996cd77 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:51.970948 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 461/927] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Oct 24 08:37:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:51.977611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899e212-5be3-42fd-b768-5d0e8f8f460c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:52.091291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899e212-5be3-42fd-b768-5d0e8f8f460c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:52.129155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f163-0f2c-4165-bb88-bf74eef0862d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:52.129860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f163-0f2c-4165-bb88-bf74eef0862d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:52.130207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f163-0f2c-4165-bb88-bf74eef0862d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:52.130481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3f163-0f2c-4165-bb88-bf74eef0862d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:52.140377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3f163-0f2c-4165-bb88-bf74eef0862d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:52.141867 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3f163-0f2c-4165-bb88-bf74eef0862d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-1af3f163-0f2c-4165-bb88-bf74eef0862d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:37:52.156457 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:52.157483 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:37:52.158407 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:37:52.158407 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:52.158407 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:37:52.158407 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 467/928] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:52.164210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282acd8-1185-4da4-ac9d-9b65c996cd77 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:52.277380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282acd8-1185-4da4-ac9d-9b65c996cd77 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:52.295300 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:52.319525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77cb0e-df94-4731-bdbd-919fdb31f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:52.319525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77cb0e-df94-4731-bdbd-919fdb31f683 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:52.319525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77cb0e-df94-4731-bdbd-919fdb31f683 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:52.319525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77cb0e-df94-4731-bdbd-919fdb31f683 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:52.320434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77cb0e-df94-4731-bdbd-919fdb31f683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:52.320434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77cb0e-df94-4731-bdbd-919fdb31f683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:52.320434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77cb0e-df94-4731-bdbd-919fdb31f683 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:52.333753 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 462/929] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:52.339804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f163-0f2c-4165-bb88-bf74eef0862d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:52.390816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:52.443926 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3f163-0f2c-4165-bb88-bf74eef0862d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:52.478562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88205f-bc7c-468b-8b87-b0d5fb534c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:52.479300 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88205f-bc7c-468b-8b87-b0d5fb534c78 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:52.479573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88205f-bc7c-468b-8b87-b0d5fb534c78 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:52.479839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88205f-bc7c-468b-8b87-b0d5fb534c78 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:52.487808 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88205f-bc7c-468b-8b87-b0d5fb534c78 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:52.489246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88205f-bc7c-468b-8b87-b0d5fb534c78 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2d88205f-bc7c-468b-8b87-b0d5fb534c78 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:37:52.500309 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:52.501802 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:37:52.503056 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 468/930] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:52.507559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77cb0e-df94-4731-bdbd-919fdb31f683 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:52.610940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77cb0e-df94-4731-bdbd-919fdb31f683 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:52.644835 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4cebe-fc40-4dda-928b-6783921d83dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:52.645495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4cebe-fc40-4dda-928b-6783921d83dc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:52.645780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4cebe-fc40-4dda-928b-6783921d83dc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:52.646019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4cebe-fc40-4dda-928b-6783921d83dc None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:52.646763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4cebe-fc40-4dda-928b-6783921d83dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:52.647064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4cebe-fc40-4dda-928b-6783921d83dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:52.648511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4cebe-fc40-4dda-928b-6783921d83dc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:52.666686 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 463/931] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Oct 24 08:37:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:52.673659 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88205f-bc7c-468b-8b87-b0d5fb534c78 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:52.782797 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88205f-bc7c-468b-8b87-b0d5fb534c78 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:52.795145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:52.821512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3f790-82ac-4bd6-91be-98da20df061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:52.822174 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3f790-82ac-4bd6-91be-98da20df061f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:52.822451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3f790-82ac-4bd6-91be-98da20df061f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:52.822734 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3f790-82ac-4bd6-91be-98da20df061f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:52.898400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3f790-82ac-4bd6-91be-98da20df061f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:52.899894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3f790-82ac-4bd6-91be-98da20df061f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:53.010264 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 469/932] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:37:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 08:37:53.018083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4cebe-fc40-4dda-928b-6783921d83dc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:53.058883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:53.133991 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4cebe-fc40-4dda-928b-6783921d83dc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:53.173731 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a7b1e-0802-4739-8c38-ebc15074e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:53.174466 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a7b1e-0802-4739-8c38-ebc15074e5e6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:53.174866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a7b1e-0802-4739-8c38-ebc15074e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:53.175233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a7b1e-0802-4739-8c38-ebc15074e5e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:53.185373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a7b1e-0802-4739-8c38-ebc15074e5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:53.187121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a7b1e-0802-4739-8c38-ebc15074e5e6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-2d6a7b1e-0802-4739-8c38-ebc15074e5e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:37:53.203044 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:53.204046 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:37:53.204935 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:37:53.204935 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:53.204935 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:37:53.204935 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 464/933] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:53.210475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3f790-82ac-4bd6-91be-98da20df061f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:53.222913 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:53.320172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3f790-82ac-4bd6-91be-98da20df061f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:53.354441 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706cc56-d954-4c9b-abad-802d8f4c3202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:53.355147 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706cc56-d954-4c9b-abad-802d8f4c3202 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:53.355351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706cc56-d954-4c9b-abad-802d8f4c3202 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:53.355618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706cc56-d954-4c9b-abad-802d8f4c3202 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:53.356252 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706cc56-d954-4c9b-abad-802d8f4c3202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:53.356547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706cc56-d954-4c9b-abad-802d8f4c3202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:53.357884 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706cc56-d954-4c9b-abad-802d8f4c3202 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:53.369230 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 470/934] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:53.375556 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a7b1e-0802-4739-8c38-ebc15074e5e6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:53.471941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:53.487363 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a7b1e-0802-4739-8c38-ebc15074e5e6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:53.520977 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750af1b-02fe-4236-9b6c-71659db40c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:53.521659 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750af1b-02fe-4236-9b6c-71659db40c94 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:53.521941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750af1b-02fe-4236-9b6c-71659db40c94 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:53.522199 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750af1b-02fe-4236-9b6c-71659db40c94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:53.529078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750af1b-02fe-4236-9b6c-71659db40c94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:53.530506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750af1b-02fe-4236-9b6c-71659db40c94 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1750af1b-02fe-4236-9b6c-71659db40c94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:37:53.543545 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:53.544552 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:53.545637 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:37:53.545637 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:37:53.545637 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:37:53.545637 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:37:53.545637 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:37:53.545637 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:37:53.545637 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:37:53.545637 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:37:53.545637 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 465/935] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:37:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:53.552986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706cc56-d954-4c9b-abad-802d8f4c3202 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:53.624655 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:53.657986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706cc56-d954-4c9b-abad-802d8f4c3202 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:53.694445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dae44-4238-474e-a854-99cf1d0f02c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:53.695085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dae44-4238-474e-a854-99cf1d0f02c1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:53.695362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dae44-4238-474e-a854-99cf1d0f02c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:53.695632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dae44-4238-474e-a854-99cf1d0f02c1 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:53.696324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dae44-4238-474e-a854-99cf1d0f02c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:53.696666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dae44-4238-474e-a854-99cf1d0f02c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:53.698332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dae44-4238-474e-a854-99cf1d0f02c1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:53.711355 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 471/936] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:37:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:53.718328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750af1b-02fe-4236-9b6c-71659db40c94 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:53.828202 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750af1b-02fe-4236-9b6c-71659db40c94 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:53.861395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8c011-c35b-45dd-8cf0-5cb453d30a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:53.862101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8c011-c35b-45dd-8cf0-5cb453d30a81 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:53.862436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8c011-c35b-45dd-8cf0-5cb453d30a81 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:53.862797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8c011-c35b-45dd-8cf0-5cb453d30a81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:53.863577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8c011-c35b-45dd-8cf0-5cb453d30a81 None admin] RBAC: Adding query filter params (role.id=cb7d27022ded4ea2afd3d5ebaefbf429, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=c3749568726644be9ebdf4d87463ac0e) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:53.863948 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8c011-c35b-45dd-8cf0-5cb453d30a81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:53.865422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8c011-c35b-45dd-8cf0-5cb453d30a81 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:53.871253 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 466/937] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:37:53 2022] GET /identity/v3/role_assignments?user.id=c3749568726644be9ebdf4d87463ac0e&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=cb7d27022ded4ea2afd3d5ebaefbf429 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:56.089863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb52ec7-f63a-4899-a9e5-315e45e680f9 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:56.090438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb52ec7-f63a-4899-a9e5-315e45e680f9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:56.090438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb52ec7-f63a-4899-a9e5-315e45e680f9 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:56.091827 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 472/938] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:37:56.097396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ebc41-2ded-47ef-8865-1be6412de74d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:56.097850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ebc41-2ded-47ef-8865-1be6412de74d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:56.097942 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ebc41-2ded-47ef-8865-1be6412de74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:56.119344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d58ebc41-2ded-47ef-8865-1be6412de74d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:37:56.120955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d58ebc41-2ded-47ef-8865-1be6412de74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:37:56.125454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:56.209981 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 467/939] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:37:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:37:56.493675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb52ec7-f63a-4899-a9e5-315e45e680f9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:56.609952 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb52ec7-f63a-4899-a9e5-315e45e680f9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:56.641008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:56.647410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955e050-90eb-4b41-ad83-6838590fc778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:56.648109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955e050-90eb-4b41-ad83-6838590fc778 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:56.648306 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955e050-90eb-4b41-ad83-6838590fc778 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:56.648602 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955e050-90eb-4b41-ad83-6838590fc778 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:56.659668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955e050-90eb-4b41-ad83-6838590fc778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7955e050-90eb-4b41-ad83-6838590fc778 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:37:56.664752 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:56.665923 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:37:56.666944 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 473/940] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:37:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:56.672626 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ebc41-2ded-47ef-8865-1be6412de74d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:56.781684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ebc41-2ded-47ef-8865-1be6412de74d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:56.818343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c583983-00ef-40e6-a934-5e3833fce26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:56.819182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c583983-00ef-40e6-a934-5e3833fce26e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:56.819557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c583983-00ef-40e6-a934-5e3833fce26e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:56.819924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c583983-00ef-40e6-a934-5e3833fce26e None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:56.820810 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c583983-00ef-40e6-a934-5e3833fce26e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:56.821146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c583983-00ef-40e6-a934-5e3833fce26e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:56.822776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c583983-00ef-40e6-a934-5e3833fce26e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:56.831183 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 468/941] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:37:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:56.837836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955e050-90eb-4b41-ad83-6838590fc778 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:56.949710 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955e050-90eb-4b41-ad83-6838590fc778 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:56.988435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb6255-0a32-457e-95e5-8c8033ec2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:56.989073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb6255-0a32-457e-95e5-8c8033ec2eb8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:56.989350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb6255-0a32-457e-95e5-8c8033ec2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:56.989619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb6255-0a32-457e-95e5-8c8033ec2eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:56.999894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb6255-0a32-457e-95e5-8c8033ec2eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:57.001330 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb6255-0a32-457e-95e5-8c8033ec2eb8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-66eb6255-0a32-457e-95e5-8c8033ec2eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:37:57.016569 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:57.017960 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:37:57.019088 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:37:57.019088 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:57.019088 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:37:57.019088 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 474/942] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:57.024234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c583983-00ef-40e6-a934-5e3833fce26e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:57.028163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:57.141502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c583983-00ef-40e6-a934-5e3833fce26e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:57.178866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaf4a8-633f-4e4f-9f5f-d971f77ecf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:57.179675 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaf4a8-633f-4e4f-9f5f-d971f77ecf13 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:57.179996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaf4a8-633f-4e4f-9f5f-d971f77ecf13 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:57.180427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaf4a8-633f-4e4f-9f5f-d971f77ecf13 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:57.180992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaf4a8-633f-4e4f-9f5f-d971f77ecf13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:57.181391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaf4a8-633f-4e4f-9f5f-d971f77ecf13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:57.182970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaf4a8-633f-4e4f-9f5f-d971f77ecf13 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:57.195732 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 469/943] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:57.202238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb6255-0a32-457e-95e5-8c8033ec2eb8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:57.249661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:57.316572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb6255-0a32-457e-95e5-8c8033ec2eb8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:57.350580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbf62c-30f9-417a-a8d1-37a33be7fe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:57.351267 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbf62c-30f9-417a-a8d1-37a33be7fe08 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:57.351567 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbf62c-30f9-417a-a8d1-37a33be7fe08 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:57.351881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbf62c-30f9-417a-a8d1-37a33be7fe08 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:57.359666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbf62c-30f9-417a-a8d1-37a33be7fe08 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:57.361106 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbf62c-30f9-417a-a8d1-37a33be7fe08 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2bbbf62c-30f9-417a-a8d1-37a33be7fe08 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:37:57.375018 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:57.376215 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:37:57.377505 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 475/944] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:57.383399 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaf4a8-633f-4e4f-9f5f-d971f77ecf13 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:57.452356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:57.493671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaf4a8-633f-4e4f-9f5f-d971f77ecf13 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:57.530412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6772e-abd5-4cd6-ba1b-c1532bccb279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:57.531346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6772e-abd5-4cd6-ba1b-c1532bccb279 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:57.531623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6772e-abd5-4cd6-ba1b-c1532bccb279 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:57.531893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6772e-abd5-4cd6-ba1b-c1532bccb279 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:57.532629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6772e-abd5-4cd6-ba1b-c1532bccb279 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:57.532924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6772e-abd5-4cd6-ba1b-c1532bccb279 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:57.534343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6772e-abd5-4cd6-ba1b-c1532bccb279 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:57.554376 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 470/945] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Oct 24 08:37:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:57.560369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbf62c-30f9-417a-a8d1-37a33be7fe08 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:57.675291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbf62c-30f9-417a-a8d1-37a33be7fe08 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:57.677259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:57.715366 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b524e9-9260-4ed1-8f13-d403ee149e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:57.716137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b524e9-9260-4ed1-8f13-d403ee149e4c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:57.716548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b524e9-9260-4ed1-8f13-d403ee149e4c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:57.716824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b524e9-9260-4ed1-8f13-d403ee149e4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:57.729793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b524e9-9260-4ed1-8f13-d403ee149e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:57.731307 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b524e9-9260-4ed1-8f13-d403ee149e4c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e5b524e9-9260-4ed1-8f13-d403ee149e4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:37:57.748288 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:57.754263 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:37:57.755455 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:37:57.755455 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:37:57.755455 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:37:57.755455 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 476/946] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:57.755743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6772e-abd5-4cd6-ba1b-c1532bccb279 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:57.857031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6772e-abd5-4cd6-ba1b-c1532bccb279 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:57.877811 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:57.892504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ced25-7858-45d1-8400-7f9de3f8800e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:57.893111 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ced25-7858-45d1-8400-7f9de3f8800e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:57.893390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ced25-7858-45d1-8400-7f9de3f8800e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:57.893662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ced25-7858-45d1-8400-7f9de3f8800e None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:57.894305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ced25-7858-45d1-8400-7f9de3f8800e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:57.894608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ced25-7858-45d1-8400-7f9de3f8800e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:57.896006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ced25-7858-45d1-8400-7f9de3f8800e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:57.910029 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 471/947] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:57.916789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b524e9-9260-4ed1-8f13-d403ee149e4c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:58.023826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b524e9-9260-4ed1-8f13-d403ee149e4c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:58.060776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eff39d-221c-4748-a599-bf8f30e254a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:58.061466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eff39d-221c-4748-a599-bf8f30e254a2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:58.061761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eff39d-221c-4748-a599-bf8f30e254a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:58.062036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eff39d-221c-4748-a599-bf8f30e254a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:58.068538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eff39d-221c-4748-a599-bf8f30e254a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:58.069968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eff39d-221c-4748-a599-bf8f30e254a2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-90eff39d-221c-4748-a599-bf8f30e254a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:37:58.080816 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:37:58.081886 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:37:58.082969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:37:58.082969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:37:58.082969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:37:58.082969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:37:58.082969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:37:58.082969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:37:58.082969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:37:58.082969 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:37:58.086890 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 477/948] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:37:58.088238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ced25-7858-45d1-8400-7f9de3f8800e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:58.198474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ced25-7858-45d1-8400-7f9de3f8800e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:58.236677 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b9e44-c987-400e-aca2-f9bfd3332b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:58.237290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b9e44-c987-400e-aca2-f9bfd3332b28 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:58.237571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b9e44-c987-400e-aca2-f9bfd3332b28 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:58.237844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b9e44-c987-400e-aca2-f9bfd3332b28 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:58.238652 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b9e44-c987-400e-aca2-f9bfd3332b28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:37:58.239207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b9e44-c987-400e-aca2-f9bfd3332b28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:58.240792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b9e44-c987-400e-aca2-f9bfd3332b28 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:58.255619 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 472/949] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:37:58.261904 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eff39d-221c-4748-a599-bf8f30e254a2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:37:58.282104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:37:58.388618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eff39d-221c-4748-a599-bf8f30e254a2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:37:58.427101 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc152154-6ab2-44c7-8b48-fdc3785021b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:37:58.427801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc152154-6ab2-44c7-8b48-fdc3785021b2 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:37:58.428076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc152154-6ab2-44c7-8b48-fdc3785021b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:37:58.428397 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc152154-6ab2-44c7-8b48-fdc3785021b2 None admin] PATH_INFO: `/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/c3749568726644be9ebdf4d87463ac0e/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:37:58.430320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc152154-6ab2-44c7-8b48-fdc3785021b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0150abead5e3470d9c6cbf9f9c9e4d4a, user_id=c3749568726644be9ebdf4d87463ac0e, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:37:58.431752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc152154-6ab2-44c7-8b48-fdc3785021b2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:37:58.445581 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 478/950] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Oct 24 08:37:58 2022] PUT /identity/v3/projects/0150abead5e3470d9c6cbf9f9c9e4d4a/users/c3749568726644be9ebdf4d87463ac0e/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:38:00.580215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15abf3e-9ed7-4ee4-ac8c-7cd7a45c675a None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:00.580870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15abf3e-9ed7-4ee4-ac8c-7cd7a45c675a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:00.580870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15abf3e-9ed7-4ee4-ac8c-7cd7a45c675a None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:00.582028 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 473/951] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:38:00.587562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30a061-5bd5-49c8-8cdd-f54009cf155d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:00.588429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30a061-5bd5-49c8-8cdd-f54009cf155d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:00.588694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30a061-5bd5-49c8-8cdd-f54009cf155d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:00.608497 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4f30a061-5bd5-49c8-8cdd-f54009cf155d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:38:00.609809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4f30a061-5bd5-49c8-8cdd-f54009cf155d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:38:00.687818 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 479/952] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:38:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:38:00.948689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15abf3e-9ed7-4ee4-ac8c-7cd7a45c675a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:01.000192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:01.069059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15abf3e-9ed7-4ee4-ac8c-7cd7a45c675a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:01.106535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff1e28-813d-40a3-8eff-1673629b2d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:01.107275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff1e28-813d-40a3-8eff-1673629b2d6c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:01.107521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff1e28-813d-40a3-8eff-1673629b2d6c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:01.107845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff1e28-813d-40a3-8eff-1673629b2d6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:01.119948 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff1e28-813d-40a3-8eff-1673629b2d6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-beff1e28-813d-40a3-8eff-1673629b2d6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 08:38:01.124750 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:38:01.125882 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:01.127754 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 474/953] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Oct 24 08:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:38:01.132526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30a061-5bd5-49c8-8cdd-f54009cf155d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:01.136140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:01.247769 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30a061-5bd5-49c8-8cdd-f54009cf155d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:01.284400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27eac6b-c3f6-4a5c-8bad-89d7765e44fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:01.285023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eac6b-c3f6-4a5c-8bad-89d7765e44fd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:01.285299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eac6b-c3f6-4a5c-8bad-89d7765e44fd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:01.285602 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eac6b-c3f6-4a5c-8bad-89d7765e44fd None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:01.286232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27eac6b-c3f6-4a5c-8bad-89d7765e44fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:01.286559 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27eac6b-c3f6-4a5c-8bad-89d7765e44fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:01.288108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27eac6b-c3f6-4a5c-8bad-89d7765e44fd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:01.298386 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 480/954] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Oct 24 08:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:38:01.305079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff1e28-813d-40a3-8eff-1673629b2d6c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:01.422081 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:01.425395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff1e28-813d-40a3-8eff-1673629b2d6c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:01.465299 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fecd24-866e-4fc3-a152-59960dbf11f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:01.466418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fecd24-866e-4fc3-a152-59960dbf11f8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:01.466698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fecd24-866e-4fc3-a152-59960dbf11f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:01.467122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fecd24-866e-4fc3-a152-59960dbf11f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:01.478775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fecd24-866e-4fc3-a152-59960dbf11f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:01.480586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fecd24-866e-4fc3-a152-59960dbf11f8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-11fecd24-866e-4fc3-a152-59960dbf11f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:38:01.497515 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:38:01.498660 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:38:01.499606 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:38:01.499606 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:38:01.499606 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:01.499606 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 475/955] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:38:01.505163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27eac6b-c3f6-4a5c-8bad-89d7765e44fd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:01.579736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:01.616461 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27eac6b-c3f6-4a5c-8bad-89d7765e44fd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:01.652821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795187f-9309-4014-9397-b672f0542434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:01.653532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795187f-9309-4014-9397-b672f0542434 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:01.653813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795187f-9309-4014-9397-b672f0542434 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:01.654081 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795187f-9309-4014-9397-b672f0542434 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:01.654749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795187f-9309-4014-9397-b672f0542434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:01.655045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795187f-9309-4014-9397-b672f0542434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:01.656412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795187f-9309-4014-9397-b672f0542434 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:01.668992 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 481/956] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:38:01.674890 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fecd24-866e-4fc3-a152-59960dbf11f8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:01.789753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fecd24-866e-4fc3-a152-59960dbf11f8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:01.823676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d5eec-6060-4675-9d01-ac36bf3ae0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:01.824300 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d5eec-6060-4675-9d01-ac36bf3ae0ef None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:01.824579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d5eec-6060-4675-9d01-ac36bf3ae0ef None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:01.824868 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d5eec-6060-4675-9d01-ac36bf3ae0ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:01.833789 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d5eec-6060-4675-9d01-ac36bf3ae0ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:01.835473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d5eec-6060-4675-9d01-ac36bf3ae0ef None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-108d5eec-6060-4675-9d01-ac36bf3ae0ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 08:38:01.849223 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:38:01.850403 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:01.851608 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 476/957] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:38:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:38:01.856986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795187f-9309-4014-9397-b672f0542434 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:01.954767 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795187f-9309-4014-9397-b672f0542434 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:01.987231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:01.998790 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ee239-3f79-4bbd-b473-b4e2d9dbdece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:01.999610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ee239-3f79-4bbd-b473-b4e2d9dbdece None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:02.000007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ee239-3f79-4bbd-b473-b4e2d9dbdece None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:02.000290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ee239-3f79-4bbd-b473-b4e2d9dbdece None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:02.001298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ee239-3f79-4bbd-b473-b4e2d9dbdece None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:02.001658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ee239-3f79-4bbd-b473-b4e2d9dbdece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:02.003401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ee239-3f79-4bbd-b473-b4e2d9dbdece None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:02.025091 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 482/958] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Oct 24 08:38:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:38:02.031522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d5eec-6060-4675-9d01-ac36bf3ae0ef None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:02.054846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:02.153759 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d5eec-6060-4675-9d01-ac36bf3ae0ef None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:02.192808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b1735-32f7-4349-937b-d0853e6ef5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:02.193416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b1735-32f7-4349-937b-d0853e6ef5e9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:02.193698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b1735-32f7-4349-937b-d0853e6ef5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:02.193972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b1735-32f7-4349-937b-d0853e6ef5e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:02.270820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b1735-32f7-4349-937b-d0853e6ef5e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:02.272406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b1735-32f7-4349-937b-d0853e6ef5e9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:02.287272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:02.385593 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 477/959] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Oct 24 08:38:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 24 08:38:02.392232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ee239-3f79-4bbd-b473-b4e2d9dbdece None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:02.494094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ee239-3f79-4bbd-b473-b4e2d9dbdece None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:02.527661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8a00e-0c61-4efd-870d-46826aa8cade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:02.528299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8a00e-0c61-4efd-870d-46826aa8cade None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:02.528573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8a00e-0c61-4efd-870d-46826aa8cade None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:02.528841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8a00e-0c61-4efd-870d-46826aa8cade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:02.539687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8a00e-0c61-4efd-870d-46826aa8cade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:02.541152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8a00e-0c61-4efd-870d-46826aa8cade None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-24f8a00e-0c61-4efd-870d-46826aa8cade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 08:38:02.556310 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:38:02.557542 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 08:38:02.558615 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 08:38:02.558615 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 08:38:02.558615 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:38:02.558615 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 483/960] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:38:02.563422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b1735-32f7-4349-937b-d0853e6ef5e9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:02.661857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b1735-32f7-4349-937b-d0853e6ef5e9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:02.676783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:02.700408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214015d-c489-4cfa-bdce-351169d344a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:02.701014 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214015d-c489-4cfa-bdce-351169d344a5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:02.701285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214015d-c489-4cfa-bdce-351169d344a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:02.701552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214015d-c489-4cfa-bdce-351169d344a5 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:02.702186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214015d-c489-4cfa-bdce-351169d344a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:02.702535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214015d-c489-4cfa-bdce-351169d344a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:02.704121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214015d-c489-4cfa-bdce-351169d344a5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:02.715518 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 478/961] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Oct 24 08:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:38:02.721801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8a00e-0c61-4efd-870d-46826aa8cade None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:02.760180 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:02.830181 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8a00e-0c61-4efd-870d-46826aa8cade None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:02.866902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cee059-0f54-4414-b397-b2d6683730ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:02.867609 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cee059-0f54-4414-b397-b2d6683730ab None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:02.867943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cee059-0f54-4414-b397-b2d6683730ab None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:02.868333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cee059-0f54-4414-b397-b2d6683730ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:02.877067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cee059-0f54-4414-b397-b2d6683730ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:02.884429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cee059-0f54-4414-b397-b2d6683730ab None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-d8cee059-0f54-4414-b397-b2d6683730ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 08:38:02.898013 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:38:02.899146 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:38:02.900275 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 08:38:02.900275 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 08:38:02.900275 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 08:38:02.900275 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 08:38:02.900275 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 08:38:02.900275 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 08:38:02.900275 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 08:38:02.900275 dv-cinder-master-3-smb devstack@keystone.service[87873]: ERROR keystone.server.flask.application Oct 24 08:38:02.902943 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 484/962] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Oct 24 08:38:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:38:02.905952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214015d-c489-4cfa-bdce-351169d344a5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:03.026533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214015d-c489-4cfa-bdce-351169d344a5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:03.067139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e2139-08ca-4980-a6fc-fe44415953c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:03.067783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2139-08ca-4980-a6fc-fe44415953c8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:03.068027 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2139-08ca-4980-a6fc-fe44415953c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:03.068273 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2139-08ca-4980-a6fc-fe44415953c8 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:03.069042 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2139-08ca-4980-a6fc-fe44415953c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:03.069370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2139-08ca-4980-a6fc-fe44415953c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:03.070947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2139-08ca-4980-a6fc-fe44415953c8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:03.084378 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 479/963] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Oct 24 08:38:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:38:03.091572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cee059-0f54-4414-b397-b2d6683730ab None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:03.193128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:03.199849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cee059-0f54-4414-b397-b2d6683730ab None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:03.237538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db82bb-6c36-44f1-beb3-9ad2c84ee77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:03.238273 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db82bb-6c36-44f1-beb3-9ad2c84ee77c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:03.238659 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db82bb-6c36-44f1-beb3-9ad2c84ee77c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:03.239016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db82bb-6c36-44f1-beb3-9ad2c84ee77c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:03.239792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db82bb-6c36-44f1-beb3-9ad2c84ee77c None admin] RBAC: Adding query filter params (role.id=eadba215d7ee4202b9fb56ea79c33b11, scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a, user.id=c3749568726644be9ebdf4d87463ac0e) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:03.240214 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db82bb-6c36-44f1-beb3-9ad2c84ee77c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:03.241951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db82bb-6c36-44f1-beb3-9ad2c84ee77c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:03.249450 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 485/964] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Oct 24 08:38:03 2022] GET /identity/v3/role_assignments?user.id=c3749568726644be9ebdf4d87463ac0e&scope.project.id=0150abead5e3470d9c6cbf9f9c9e4d4a&role.id=eadba215d7ee4202b9fb56ea79c33b11 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:38:05.379726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174479ed-7750-4c4d-ad24-e67a04e9e659 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:05.380343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174479ed-7750-4c4d-ad24-e67a04e9e659 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:05.380343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174479ed-7750-4c4d-ad24-e67a04e9e659 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:05.381582 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 480/965] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:38:05.388226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6cec9-607b-48d5-a78e-10d4a9cf20f4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:05.388909 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6cec9-607b-48d5-a78e-10d4a9cf20f4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:05.389150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6cec9-607b-48d5-a78e-10d4a9cf20f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:05.419231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c2a6cec9-607b-48d5-a78e-10d4a9cf20f4 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:38:05.420429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c2a6cec9-607b-48d5-a78e-10d4a9cf20f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:38:05.489194 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 486/966] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:38:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:38:05.755125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174479ed-7750-4c4d-ad24-e67a04e9e659 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:05.789719 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:05.864153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174479ed-7750-4c4d-ad24-e67a04e9e659 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:05.903292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2991321-a387-428c-90dd-a43746065af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:05.904168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2991321-a387-428c-90dd-a43746065af6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:05.904510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2991321-a387-428c-90dd-a43746065af6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:05.904851 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2991321-a387-428c-90dd-a43746065af6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:05.905721 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2991321-a387-428c-90dd-a43746065af6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:05.907916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2991321-a387-428c-90dd-a43746065af6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e2991321-a387-428c-90dd-a43746065af6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:38:05.920668 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:38:05.921851 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:05.922978 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 481/967] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Oct 24 08:38:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:38:05.928325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6cec9-607b-48d5-a78e-10d4a9cf20f4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:06.034142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:06.048394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6cec9-607b-48d5-a78e-10d4a9cf20f4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:06.089049 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d1fa6-7606-4015-bbd3-0b714af6c240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:06.089658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1fa6-7606-4015-bbd3-0b714af6c240 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:06.089937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1fa6-7606-4015-bbd3-0b714af6c240 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:06.090203 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1fa6-7606-4015-bbd3-0b714af6c240 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:06.090847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d1fa6-7606-4015-bbd3-0b714af6c240 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:06.091145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d1fa6-7606-4015-bbd3-0b714af6c240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:06.092546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d1fa6-7606-4015-bbd3-0b714af6c240 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:06.097670 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 487/968] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Oct 24 08:38:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:38:06.103781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2991321-a387-428c-90dd-a43746065af6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:06.203531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:06.209844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2991321-a387-428c-90dd-a43746065af6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:06.243837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2853d-a802-4199-8d33-dcd693d2964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:06.244421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2853d-a802-4199-8d33-dcd693d2964c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:06.244745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2853d-a802-4199-8d33-dcd693d2964c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:06.245018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2853d-a802-4199-8d33-dcd693d2964c None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:06.245628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2853d-a802-4199-8d33-dcd693d2964c None admin] RBAC: Adding query filter params (type=placement) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:06.245930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2853d-a802-4199-8d33-dcd693d2964c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:06.247341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2853d-a802-4199-8d33-dcd693d2964c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:06.251817 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 482/969] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Oct 24 08:38:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:38:08.375167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cf6da-3114-42ce-a02f-a987715d8b8b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:08.376072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cf6da-3114-42ce-a02f-a987715d8b8b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:08.376072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cf6da-3114-42ce-a02f-a987715d8b8b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:08.378124 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 488/970] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:38:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:38:08.386246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15337-3838-4fd0-920c-5d2473fd6369 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:08.386699 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15337-3838-4fd0-920c-5d2473fd6369 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:08.386823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15337-3838-4fd0-920c-5d2473fd6369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:08.410542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1bd15337-3838-4fd0-920c-5d2473fd6369 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:38:08.412341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1bd15337-3838-4fd0-920c-5d2473fd6369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:38:08.496634 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 483/971] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:38:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:38:08.767050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cf6da-3114-42ce-a02f-a987715d8b8b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:08.878332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cf6da-3114-42ce-a02f-a987715d8b8b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:08.912436 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3553d2-352e-4b0b-adc1-106595851a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:08.913026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3553d2-352e-4b0b-adc1-106595851a95 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:08.913640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3553d2-352e-4b0b-adc1-106595851a95 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:08.913919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3553d2-352e-4b0b-adc1-106595851a95 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:08.914558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3553d2-352e-4b0b-adc1-106595851a95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:08.915954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3553d2-352e-4b0b-adc1-106595851a95 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:08.933901 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 489/972] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Oct 24 08:38:08 2022] POST /identity/v3/services => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:38:11.043012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6e12c-648e-454b-929b-f51fc5708e14 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:11.043698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6e12c-648e-454b-929b-f51fc5708e14 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:11.043698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6e12c-648e-454b-929b-f51fc5708e14 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:11.044920 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 484/973] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:38:11.050381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0246-fb09-4f6f-9aa0-e6bbb4837499 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:11.050914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0246-fb09-4f6f-9aa0-e6bbb4837499 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:11.051023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0246-fb09-4f6f-9aa0-e6bbb4837499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:11.069881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e31b0246-fb09-4f6f-9aa0-e6bbb4837499 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:38:11.071201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e31b0246-fb09-4f6f-9aa0-e6bbb4837499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:38:11.080530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:11.150733 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 490/974] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:38:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 08:38:11.421997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6e12c-648e-454b-929b-f51fc5708e14 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:11.525130 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:11.536676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6e12c-648e-454b-929b-f51fc5708e14 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:11.571630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579d4d5-225c-44c3-aaff-671e4a715538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:11.572240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579d4d5-225c-44c3-aaff-671e4a715538 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:11.572511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579d4d5-225c-44c3-aaff-671e4a715538 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:11.572804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579d4d5-225c-44c3-aaff-671e4a715538 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:11.573373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579d4d5-225c-44c3-aaff-671e4a715538 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:11.574758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579d4d5-225c-44c3-aaff-671e4a715538 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d579d4d5-225c-44c3-aaff-671e4a715538 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:38:11.583247 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:38:11.584421 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:11.585368 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 485/975] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Oct 24 08:38:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:38:11.590750 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0246-fb09-4f6f-9aa0-e6bbb4837499 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:11.693262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0246-fb09-4f6f-9aa0-e6bbb4837499 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:11.727452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188896-e152-43a2-803a-c3d711e95acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:11.728042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188896-e152-43a2-803a-c3d711e95acb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:11.728314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188896-e152-43a2-803a-c3d711e95acb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:11.728581 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188896-e152-43a2-803a-c3d711e95acb None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:11.729204 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd188896-e152-43a2-803a-c3d711e95acb None admin] RBAC: Adding query filter params (name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:11.729495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd188896-e152-43a2-803a-c3d711e95acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:11.730886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd188896-e152-43a2-803a-c3d711e95acb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:11.736114 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 491/976] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Oct 24 08:38:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:38:11.742150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579d4d5-225c-44c3-aaff-671e4a715538 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:11.845865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579d4d5-225c-44c3-aaff-671e4a715538 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:11.880079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d9f5a-2153-4115-9dba-16d25ec04244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:11.880653 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d9f5a-2153-4115-9dba-16d25ec04244 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:11.880922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d9f5a-2153-4115-9dba-16d25ec04244 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:11.881189 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d9f5a-2153-4115-9dba-16d25ec04244 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:11.881876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d9f5a-2153-4115-9dba-16d25ec04244 None admin] RBAC: Adding query filter params (interface=public, service_id=b603ffb130054fb1848faf164eb2e2f6, region_id=RegionOne) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:11.882177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d9f5a-2153-4115-9dba-16d25ec04244 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:11.883549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d9f5a-2153-4115-9dba-16d25ec04244 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:11.888639 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 486/977] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Oct 24 08:38:11 2022] GET /identity/v3/endpoints?service_id=b603ffb130054fb1848faf164eb2e2f6&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:38:11.894585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188896-e152-43a2-803a-c3d711e95acb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:11.898525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:12.006682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188896-e152-43a2-803a-c3d711e95acb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:12.044814 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adbb6e-81bf-4752-b98d-23e3454aa126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:12.045418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adbb6e-81bf-4752-b98d-23e3454aa126 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:12.045692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adbb6e-81bf-4752-b98d-23e3454aa126 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:12.045961 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adbb6e-81bf-4752-b98d-23e3454aa126 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:12.046559 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adbb6e-81bf-4752-b98d-23e3454aa126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:12.047988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adbb6e-81bf-4752-b98d-23e3454aa126 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:12.055009 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 492/978] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Oct 24 08:38:11 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:38:14.179704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553628-d9f5-4a4b-a884-1b060d43c41b None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:14.180568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553628-d9f5-4a4b-a884-1b060d43c41b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:14.180568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553628-d9f5-4a4b-a884-1b060d43c41b None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:14.182652 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 487/979] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:38:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:38:14.188945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb66b0-b418-404e-a210-1f21d0a79b20 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:14.189384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb66b0-b418-404e-a210-1f21d0a79b20 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:14.189524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb66b0-b418-404e-a210-1f21d0a79b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:14.212875 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-63cb66b0-b418-404e-a210-1f21d0a79b20 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:38:14.214057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-63cb66b0-b418-404e-a210-1f21d0a79b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:38:14.283572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:14.291143 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 493/980] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:38:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 08:38:14.556162 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553628-d9f5-4a4b-a884-1b060d43c41b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:14.590298 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:14.676389 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553628-d9f5-4a4b-a884-1b060d43c41b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:14.708953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6f6c-2ccc-4e2e-9a3e-30cd308cdc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:14.709579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6f6c-2ccc-4e2e-9a3e-30cd308cdc5e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:14.709857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6f6c-2ccc-4e2e-9a3e-30cd308cdc5e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:14.710131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6f6c-2ccc-4e2e-9a3e-30cd308cdc5e None admin] PATH_INFO: `/v3/services/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:14.710709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c6f6c-2ccc-4e2e-9a3e-30cd308cdc5e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:14.712110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c6f6c-2ccc-4e2e-9a3e-30cd308cdc5e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-4e2c6f6c-2ccc-4e2e-9a3e-30cd308cdc5e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 08:38:14.720971 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 08:38:14.722111 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: ERROR keystone.server.flask.application Oct 24 08:38:14.723326 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 488/981] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Oct 24 08:38:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 08:38:14.728291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb66b0-b418-404e-a210-1f21d0a79b20 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:14.827636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb66b0-b418-404e-a210-1f21d0a79b20 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:14.869671 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba0b3c-0cb1-4cfb-9973-f035fca53343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:14.870284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba0b3c-0cb1-4cfb-9973-f035fca53343 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:14.870561 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba0b3c-0cb1-4cfb-9973-f035fca53343 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:14.870853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba0b3c-0cb1-4cfb-9973-f035fca53343 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:14.871475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba0b3c-0cb1-4cfb-9973-f035fca53343 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:38:14.871782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba0b3c-0cb1-4cfb-9973-f035fca53343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:14.873141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba0b3c-0cb1-4cfb-9973-f035fca53343 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:14.878378 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 494/982] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Oct 24 08:38:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:38:14.884213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6f6c-2ccc-4e2e-9a3e-30cd308cdc5e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:38:14.974808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:14.994375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6f6c-2ccc-4e2e-9a3e-30cd308cdc5e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:38:15.034764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052eef5-934e-4904-9d2b-2b08e87858c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:38:15.035472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052eef5-934e-4904-9d2b-2b08e87858c6 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:38:15.035815 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052eef5-934e-4904-9d2b-2b08e87858c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:38:15.036137 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052eef5-934e-4904-9d2b-2b08e87858c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:38:15.036913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052eef5-934e-4904-9d2b-2b08e87858c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:38:15.038979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052eef5-934e-4904-9d2b-2b08e87858c6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:38:15.062749 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 489/983] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:38:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:38:24.875212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=69 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:38:25.057094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:18.467407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6beeb2-6991-4097-80ed-cc88a0481055 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:18.467892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6beeb2-6991-4097-80ed-cc88a0481055 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:18.467980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6beeb2-6991-4097-80ed-cc88a0481055 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:18.471513 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 495/984] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:18.474669 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c30ba-11b4-4e85-91e5-72c14cc99790 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:18.475144 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c30ba-11b4-4e85-91e5-72c14cc99790 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:18.475245 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c30ba-11b4-4e85-91e5-72c14cc99790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:18.574287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c29c30ba-11b4-4e85-91e5-72c14cc99790 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:18.576189 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c29c30ba-11b4-4e85-91e5-72c14cc99790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:18.697320 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 490/985] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:18.990053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe337c-1f0f-4acb-9121-48fb99116adf None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:18.990405 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe337c-1f0f-4acb-9121-48fb99116adf None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:18.990598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe337c-1f0f-4acb-9121-48fb99116adf None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:18.992216 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 496/986] 10.222.0.26 () {54 vars in 964 bytes} [Mon Oct 24 08:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:18.998505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b934d16-5a44-4bb7-91e1-ceb06235237f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:18.998824 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b934d16-5a44-4bb7-91e1-ceb06235237f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:18.999089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b934d16-5a44-4bb7-91e1-ceb06235237f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:19.085205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1b934d16-5a44-4bb7-91e1-ceb06235237f None None] MFA Rules not processed for user `2254623f523c48f0aa454b85be41df95`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:19.086894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1b934d16-5a44-4bb7-91e1-ceb06235237f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:19.112445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:19.175296 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 491/987] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Oct 24 08:39:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:19.184601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe337c-1f0f-4acb-9121-48fb99116adf None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:19.186745 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-58fe337c-1f0f-4acb-9121-48fb99116adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:19.321018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe337c-1f0f-4acb-9121-48fb99116adf None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:19.404859 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520aee1-ecb7-4f39-869f-84bc5e561584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:19.405481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520aee1-ecb7-4f39-869f-84bc5e561584 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:19.405753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520aee1-ecb7-4f39-869f-84bc5e561584 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:19.406022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520aee1-ecb7-4f39-869f-84bc5e561584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:19.407668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f520aee1-ecb7-4f39-869f-84bc5e561584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:19.465050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:19.524698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520aee1-ecb7-4f39-869f-84bc5e561584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:19.526151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520aee1-ecb7-4f39-869f-84bc5e561584 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:19.557309 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 497/988] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:22.028740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c74fb0-b422-427e-a086-9ee4b6bc9ae4 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:22.029314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c74fb0-b422-427e-a086-9ee4b6bc9ae4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:22.029314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c74fb0-b422-427e-a086-9ee4b6bc9ae4 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:22.030611 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 492/989] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:39:22.036256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fa426-6aa4-4bad-84bf-50662cf40e8d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:22.036724 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fa426-6aa4-4bad-84bf-50662cf40e8d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:22.036854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fa426-6aa4-4bad-84bf-50662cf40e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:22.110533 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fe4fa426-6aa4-4bad-84bf-50662cf40e8d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:22.111794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fe4fa426-6aa4-4bad-84bf-50662cf40e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:22.211753 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 498/990] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:22.494048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c74fb0-b422-427e-a086-9ee4b6bc9ae4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:22.496578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-05c74fb0-b422-427e-a086-9ee4b6bc9ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:22.612786 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c74fb0-b422-427e-a086-9ee4b6bc9ae4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:22.703338 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:22.706197 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a8d6d-bbcf-4701-b5ce-6ec8a6e3b632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:22.706837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a8d6d-bbcf-4701-b5ce-6ec8a6e3b632 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:22.707128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a8d6d-bbcf-4701-b5ce-6ec8a6e3b632 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:22.707570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a8d6d-bbcf-4701-b5ce-6ec8a6e3b632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:22.710140 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4c9a8d6d-bbcf-4701-b5ce-6ec8a6e3b632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:22.845850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a8d6d-bbcf-4701-b5ce-6ec8a6e3b632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:22.847266 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a8d6d-bbcf-4701-b5ce-6ec8a6e3b632 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:22.866693 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 493/991] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:25.346042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67386a-6630-4c05-a0f2-d0c0e87702df None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:25.346631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67386a-6630-4c05-a0f2-d0c0e87702df None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:25.346631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67386a-6630-4c05-a0f2-d0c0e87702df None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:25.348024 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 499/992] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:25.354542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4644ecd-ba30-42c7-b9fe-9ea522008875 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:25.355383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4644ecd-ba30-42c7-b9fe-9ea522008875 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:25.355476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4644ecd-ba30-42c7-b9fe-9ea522008875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:25.422822 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f4644ecd-ba30-42c7-b9fe-9ea522008875 None None] MFA Rules not processed for user `5acfe6e3e7be4573aab3cde32be1d7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:25.424033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f4644ecd-ba30-42c7-b9fe-9ea522008875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:25.477024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:25.522982 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 494/993] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:25.811677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67386a-6630-4c05-a0f2-d0c0e87702df None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:25.813224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bd67386a-6630-4c05-a0f2-d0c0e87702df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:25.823689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:25.920955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67386a-6630-4c05-a0f2-d0c0e87702df None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:25.996370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbd6a3-95c5-4294-b132-2dc9ede7de03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:25.996972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbd6a3-95c5-4294-b132-2dc9ede7de03 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:25.997256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbd6a3-95c5-4294-b132-2dc9ede7de03 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:25.997527 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbd6a3-95c5-4294-b132-2dc9ede7de03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:25.999220 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-27fbd6a3-95c5-4294-b132-2dc9ede7de03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:26.063418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:26.110736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbd6a3-95c5-4294-b132-2dc9ede7de03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:26.112194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbd6a3-95c5-4294-b132-2dc9ede7de03 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:26.131833 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 500/994] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:39:28.791570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e0385-8981-46ab-8ad8-0b9aefca57b9 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:28.792206 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e0385-8981-46ab-8ad8-0b9aefca57b9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:28.792206 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e0385-8981-46ab-8ad8-0b9aefca57b9 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:28.794121 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 495/995] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:28.802711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bafeb1-1b67-48d0-9e96-f8c834369a45 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:28.803446 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bafeb1-1b67-48d0-9e96-f8c834369a45 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:28.803560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bafeb1-1b67-48d0-9e96-f8c834369a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:28.882027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-53bafeb1-1b67-48d0-9e96-f8c834369a45 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:28.883289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-53bafeb1-1b67-48d0-9e96-f8c834369a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:28.982493 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 501/996] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:29.265370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e0385-8981-46ab-8ad8-0b9aefca57b9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:29.266952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ed4e0385-8981-46ab-8ad8-0b9aefca57b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:29.377638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e0385-8981-46ab-8ad8-0b9aefca57b9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:29.455916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d40d0b-a7ac-4f65-bf49-c490e4f92882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:29.456519 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d40d0b-a7ac-4f65-bf49-c490e4f92882 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:29.456797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d40d0b-a7ac-4f65-bf49-c490e4f92882 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:29.457069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d40d0b-a7ac-4f65-bf49-c490e4f92882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:29.458647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f7d40d0b-a7ac-4f65-bf49-c490e4f92882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:29.465921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:29.577608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d40d0b-a7ac-4f65-bf49-c490e4f92882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:29.578990 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d40d0b-a7ac-4f65-bf49-c490e4f92882 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:29.598112 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 496/997] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:32.383155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374e11a-3a24-499e-8da7-272e7aa265ec None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:32.383697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374e11a-3a24-499e-8da7-272e7aa265ec None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:32.383697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374e11a-3a24-499e-8da7-272e7aa265ec None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:32.385099 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 502/998] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:32.390543 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb3d6a-cb46-46dc-85cc-a752c2ba37d6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:32.391025 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb3d6a-cb46-46dc-85cc-a752c2ba37d6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:32.391103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb3d6a-cb46-46dc-85cc-a752c2ba37d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:32.466899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1cfb3d6a-cb46-46dc-85cc-a752c2ba37d6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:32.468147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1cfb3d6a-cb46-46dc-85cc-a752c2ba37d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:32.530493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:32.573925 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 497/999] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:32.863750 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcd9a5-d7d3-4916-ba15-eaf6d884a456 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:32.864134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcd9a5-d7d3-4916-ba15-eaf6d884a456 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:32.864470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcd9a5-d7d3-4916-ba15-eaf6d884a456 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:32.865858 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 503/1000] 10.222.0.26 () {54 vars in 964 bytes} [Mon Oct 24 08:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:32.871951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf4bd2-751a-47ea-930e-fe380c821412 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:32.872317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf4bd2-751a-47ea-930e-fe380c821412 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:32.872651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf4bd2-751a-47ea-930e-fe380c821412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:32.950830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-37cf4bd2-751a-47ea-930e-fe380c821412 None None] MFA Rules not processed for user `2254623f523c48f0aa454b85be41df95`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:32.952035 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-37cf4bd2-751a-47ea-930e-fe380c821412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:33.030321 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 498/1001] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Oct 24 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:33.039362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcd9a5-d7d3-4916-ba15-eaf6d884a456 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:33.041411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-10fcd9a5-d7d3-4916-ba15-eaf6d884a456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:33.062552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:33.144977 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcd9a5-d7d3-4916-ba15-eaf6d884a456 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:33.221969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cc3f9-3c17-4935-b474-9e39feba6e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:33.222583 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cc3f9-3c17-4935-b474-9e39feba6e5e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:33.222885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cc3f9-3c17-4935-b474-9e39feba6e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:33.223155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cc3f9-3c17-4935-b474-9e39feba6e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:33.224735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1b4cc3f9-3c17-4935-b474-9e39feba6e5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:33.298271 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:33.336887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cc3f9-3c17-4935-b474-9e39feba6e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:33.338340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cc3f9-3c17-4935-b474-9e39feba6e5e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:33.356672 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 504/1002] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:35.767419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8074e16-81f1-49e7-9a7b-491eb5bcc801 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:35.767983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8074e16-81f1-49e7-9a7b-491eb5bcc801 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:35.767983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8074e16-81f1-49e7-9a7b-491eb5bcc801 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:35.769363 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 499/1003] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:35.774557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a27f7-0e59-4511-b623-10e96e9ef327 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:35.775010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a27f7-0e59-4511-b623-10e96e9ef327 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:35.775260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a27f7-0e59-4511-b623-10e96e9ef327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:35.844228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ab2a27f7-0e59-4511-b623-10e96e9ef327 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:35.845421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ab2a27f7-0e59-4511-b623-10e96e9ef327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:35.944270 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 505/1004] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:36.219959 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8074e16-81f1-49e7-9a7b-491eb5bcc801 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:36.221571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d8074e16-81f1-49e7-9a7b-491eb5bcc801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:36.285393 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:36.329069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8074e16-81f1-49e7-9a7b-491eb5bcc801 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:36.405025 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9b90a-464a-4d83-9808-9d7d8bc29eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:36.405515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9b90a-464a-4d83-9808-9d7d8bc29eda service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:36.405815 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9b90a-464a-4d83-9808-9d7d8bc29eda service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:36.406084 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9b90a-464a-4d83-9808-9d7d8bc29eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:36.407730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-40f9b90a-464a-4d83-9808-9d7d8bc29eda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:36.514795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:36.517568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9b90a-464a-4d83-9808-9d7d8bc29eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:36.519488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9b90a-464a-4d83-9808-9d7d8bc29eda service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:36.538577 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 500/1005] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:38.719356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded014f2-5f79-4daa-bfc3-c1e60db5f533 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:38.719873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded014f2-5f79-4daa-bfc3-c1e60db5f533 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:38.719873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded014f2-5f79-4daa-bfc3-c1e60db5f533 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:38.721226 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 506/1006] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:38.727257 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c21d88-51e0-4cf4-add7-13837c984948 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:38.727865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c21d88-51e0-4cf4-add7-13837c984948 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:38.727865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c21d88-51e0-4cf4-add7-13837c984948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:38.800020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-53c21d88-51e0-4cf4-add7-13837c984948 None None] MFA Rules not processed for user `5acfe6e3e7be4573aab3cde32be1d7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:38.801287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-53c21d88-51e0-4cf4-add7-13837c984948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:38.906117 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 501/1007] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:39.188250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded014f2-5f79-4daa-bfc3-c1e60db5f533 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:39.189778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ded014f2-5f79-4daa-bfc3-c1e60db5f533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:39.221051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:39.295752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded014f2-5f79-4daa-bfc3-c1e60db5f533 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:39.370323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727c1a8-3233-453e-9bff-47a36e12f8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:39.371001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727c1a8-3233-453e-9bff-47a36e12f8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:39.371276 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727c1a8-3233-453e-9bff-47a36e12f8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:39.371543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727c1a8-3233-453e-9bff-47a36e12f8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:39.373108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4727c1a8-3233-453e-9bff-47a36e12f8a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:39.453507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:39.486016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727c1a8-3233-453e-9bff-47a36e12f8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:39.487452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727c1a8-3233-453e-9bff-47a36e12f8a2 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:39.505518 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 507/1008] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:41.832773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29a375-c54f-4b3b-b3ea-56ba83953bde None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:41.833404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29a375-c54f-4b3b-b3ea-56ba83953bde None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:41.833404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29a375-c54f-4b3b-b3ea-56ba83953bde None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:41.834660 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 502/1009] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:41.841132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a40c5b-59fb-4ec3-8aaa-e503b5a3762a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:41.841567 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a40c5b-59fb-4ec3-8aaa-e503b5a3762a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:41.841658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a40c5b-59fb-4ec3-8aaa-e503b5a3762a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:41.913793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-42a40c5b-59fb-4ec3-8aaa-e503b5a3762a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:41.915056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-42a40c5b-59fb-4ec3-8aaa-e503b5a3762a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:42.018618 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 508/1010] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:42.297301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29a375-c54f-4b3b-b3ea-56ba83953bde None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:42.298902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-be29a375-c54f-4b3b-b3ea-56ba83953bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:42.373501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:42.410937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29a375-c54f-4b3b-b3ea-56ba83953bde None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:42.499907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ccaab4-d270-4395-9422-35e3cac56731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:42.500818 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccaab4-d270-4395-9422-35e3cac56731 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:42.501097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccaab4-d270-4395-9422-35e3cac56731 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:42.501375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ccaab4-d270-4395-9422-35e3cac56731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:42.502977 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-63ccaab4-d270-4395-9422-35e3cac56731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:42.620862 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ccaab4-d270-4395-9422-35e3cac56731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:42.622300 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ccaab4-d270-4395-9422-35e3cac56731 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:42.632321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:42.641053 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 503/1011] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:44.813663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14ae4d-7a03-490b-abe7-66655e4ac25d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:44.814573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14ae4d-7a03-490b-abe7-66655e4ac25d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:44.814573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14ae4d-7a03-490b-abe7-66655e4ac25d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:44.816701 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 509/1012] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:44.823034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f7765-7c99-4975-ae32-0bdec43b4607 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:44.823484 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f7765-7c99-4975-ae32-0bdec43b4607 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:44.823645 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f7765-7c99-4975-ae32-0bdec43b4607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:44.898642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bf2f7765-7c99-4975-ae32-0bdec43b4607 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:44.900558 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bf2f7765-7c99-4975-ae32-0bdec43b4607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:45.013805 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 504/1013] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:45.305170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14ae4d-7a03-490b-abe7-66655e4ac25d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:45.306737 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-eb14ae4d-7a03-490b-abe7-66655e4ac25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:45.349206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:45.417670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14ae4d-7a03-490b-abe7-66655e4ac25d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:45.492897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84734cb2-06e6-450e-b643-a073b3db3eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:45.493502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84734cb2-06e6-450e-b643-a073b3db3eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:45.493753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84734cb2-06e6-450e-b643-a073b3db3eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:45.494039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84734cb2-06e6-450e-b643-a073b3db3eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:45.495618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-84734cb2-06e6-450e-b643-a073b3db3eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:45.580872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:45.607136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84734cb2-06e6-450e-b643-a073b3db3eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:45.608568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84734cb2-06e6-450e-b643-a073b3db3eb8 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:45.626757 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 510/1014] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:48.088176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc5487-8d83-4454-81c1-91fc62f2f7e6 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:48.088704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc5487-8d83-4454-81c1-91fc62f2f7e6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:48.088704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc5487-8d83-4454-81c1-91fc62f2f7e6 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:48.090029 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 505/1015] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:48.095506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289d0ad-841e-4426-873e-94275fa5323c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:48.095940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289d0ad-841e-4426-873e-94275fa5323c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:48.096040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289d0ad-841e-4426-873e-94275fa5323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:48.165302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4289d0ad-841e-4426-873e-94275fa5323c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:48.166922 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4289d0ad-841e-4426-873e-94275fa5323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:48.264146 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 511/1016] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:48.542791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc5487-8d83-4454-81c1-91fc62f2f7e6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:48.545028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8bfc5487-8d83-4454-81c1-91fc62f2f7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:48.625577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:48.653559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc5487-8d83-4454-81c1-91fc62f2f7e6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:48.731465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b750ed-2ad2-499f-a2d2-82f5cb4e284e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:48.732059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b750ed-2ad2-499f-a2d2-82f5cb4e284e service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:48.732335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b750ed-2ad2-499f-a2d2-82f5cb4e284e service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:48.732602 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b750ed-2ad2-499f-a2d2-82f5cb4e284e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:48.734176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-77b750ed-2ad2-499f-a2d2-82f5cb4e284e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:48.851986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b750ed-2ad2-499f-a2d2-82f5cb4e284e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:48.853313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b750ed-2ad2-499f-a2d2-82f5cb4e284e service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:48.873327 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 506/1017] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:53.312096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4bfe4-e759-42db-9195-fc2a27f921ce None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:53.312599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4bfe4-e759-42db-9195-fc2a27f921ce None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:53.312786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4bfe4-e759-42db-9195-fc2a27f921ce None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:53.314194 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 512/1018] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:53.319644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720cf6-c81e-42a7-b1e9-d7d6d676661a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:53.320121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720cf6-c81e-42a7-b1e9-d7d6d676661a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:53.320214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720cf6-c81e-42a7-b1e9-d7d6d676661a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:53.325944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:53.387540 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f2720cf6-c81e-42a7-b1e9-d7d6d676661a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:53.388678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f2720cf6-c81e-42a7-b1e9-d7d6d676661a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:53.494945 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 507/1019] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:53.771832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4bfe4-e759-42db-9195-fc2a27f921ce None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:53.773450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b0b4bfe4-e759-42db-9195-fc2a27f921ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:53.826635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:53.885495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4bfe4-e759-42db-9195-fc2a27f921ce None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:53.962030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91fb2d-e60c-44cf-aa06-64ea0197c85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:53.962642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91fb2d-e60c-44cf-aa06-64ea0197c85d service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:53.962950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91fb2d-e60c-44cf-aa06-64ea0197c85d service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:53.963222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91fb2d-e60c-44cf-aa06-64ea0197c85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:53.964833 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ce91fb2d-e60c-44cf-aa06-64ea0197c85d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:54.062751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:54.080693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91fb2d-e60c-44cf-aa06-64ea0197c85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:54.082142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91fb2d-e60c-44cf-aa06-64ea0197c85d service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:54.100909 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 513/1020] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:39:56.887657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe382b-0e57-4630-9957-4bf472bc81de None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:56.888228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe382b-0e57-4630-9957-4bf472bc81de None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:56.888228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe382b-0e57-4630-9957-4bf472bc81de None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:56.889570 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 508/1021] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:39:56.894914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207d2dc-0349-486d-8a62-c46875806a0f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:56.895363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207d2dc-0349-486d-8a62-c46875806a0f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:56.895481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207d2dc-0349-486d-8a62-c46875806a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:56.962546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e207d2dc-0349-486d-8a62-c46875806a0f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:39:56.963804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e207d2dc-0349-486d-8a62-c46875806a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:57.072664 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 514/1022] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:39:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:39:57.354461 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe382b-0e57-4630-9957-4bf472bc81de None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:39:57.356047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0cbe382b-0e57-4630-9957-4bf472bc81de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:57.451665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:39:57.469083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe382b-0e57-4630-9957-4bf472bc81de None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:39:57.551708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5fc31-b2da-473d-a7d1-b057d9695e6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:39:57.552228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5fc31-b2da-473d-a7d1-b057d9695e6b service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:39:57.552502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5fc31-b2da-473d-a7d1-b057d9695e6b service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:39:57.552791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5fc31-b2da-473d-a7d1-b057d9695e6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:39:57.554372 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7eb5fc31-b2da-473d-a7d1-b057d9695e6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:39:57.667553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5fc31-b2da-473d-a7d1-b057d9695e6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:39:57.669086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5fc31-b2da-473d-a7d1-b057d9695e6b service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:39:57.688284 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 509/1023] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:39:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:01.766266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584fabe-bc83-46e6-b1b0-cea3c91601bf None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:01.766848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584fabe-bc83-46e6-b1b0-cea3c91601bf None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:01.766848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584fabe-bc83-46e6-b1b0-cea3c91601bf None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:01.768188 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 515/1024] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:01.774762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5e78-5752-477a-97e0-85fb6fc9ce6c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:01.774762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5e78-5752-477a-97e0-85fb6fc9ce6c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:01.774762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5e78-5752-477a-97e0-85fb6fc9ce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:01.790440 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:01.847007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ae2d5e78-5752-477a-97e0-85fb6fc9ce6c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:01.848237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ae2d5e78-5752-477a-97e0-85fb6fc9ce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:01.947502 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 510/1025] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:02.226557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584fabe-bc83-46e6-b1b0-cea3c91601bf None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:02.228137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7584fabe-bc83-46e6-b1b0-cea3c91601bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:02.292445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:02.340323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584fabe-bc83-46e6-b1b0-cea3c91601bf None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:02.419197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f4410-4c52-4b79-b0f6-7a82cb10254e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:02.419815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f4410-4c52-4b79-b0f6-7a82cb10254e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:02.420086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f4410-4c52-4b79-b0f6-7a82cb10254e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:02.420348 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f4410-4c52-4b79-b0f6-7a82cb10254e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:02.421985 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-866f4410-4c52-4b79-b0f6-7a82cb10254e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:02.524135 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:02.528585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f4410-4c52-4b79-b0f6-7a82cb10254e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:02.530026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f4410-4c52-4b79-b0f6-7a82cb10254e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:02.550180 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 516/1026] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:07.441452 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41db81-758c-49c1-bf06-80d4fb707957 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:07.441972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41db81-758c-49c1-bf06-80d4fb707957 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:07.442089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41db81-758c-49c1-bf06-80d4fb707957 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:07.443418 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 511/1027] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:07.453132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75a93a-0b2d-471b-a7dd-26c39d7fa868 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:07.453951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75a93a-0b2d-471b-a7dd-26c39d7fa868 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:07.454371 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75a93a-0b2d-471b-a7dd-26c39d7fa868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:07.554198 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4c75a93a-0b2d-471b-a7dd-26c39d7fa868 None None] MFA Rules not processed for user `c3749568726644be9ebdf4d87463ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:07.555663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4c75a93a-0b2d-471b-a7dd-26c39d7fa868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:07.674233 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 517/1028] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:40:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:11.737594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988bf62-b90e-40df-88f6-eef749fc694e None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:11.738509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988bf62-b90e-40df-88f6-eef749fc694e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:11.738807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988bf62-b90e-40df-88f6-eef749fc694e None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:11.740383 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 512/1029] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:11.745666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368dac8c-b23a-4d83-a976-6d5123b9eeda None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:11.746263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368dac8c-b23a-4d83-a976-6d5123b9eeda None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:11.746347 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368dac8c-b23a-4d83-a976-6d5123b9eeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:11.801817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:11.827052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-368dac8c-b23a-4d83-a976-6d5123b9eeda None None] MFA Rules not processed for user `c3749568726644be9ebdf4d87463ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:11.828257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-368dac8c-b23a-4d83-a976-6d5123b9eeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:11.934312 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 518/1030] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:40:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:11.947130 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=59 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:13.251712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2110b-a6fd-4781-bfe4-5457b6356e62 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:13.252408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2110b-a6fd-4781-bfe4-5457b6356e62 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:13.252408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2110b-a6fd-4781-bfe4-5457b6356e62 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:13.253609 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 513/1031] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:13.258627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9ff53-0bb7-4adb-b9f3-b14f92ad7b09 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:13.259078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9ff53-0bb7-4adb-b9f3-b14f92ad7b09 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:13.259222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9ff53-0bb7-4adb-b9f3-b14f92ad7b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:13.335114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-68d9ff53-0bb7-4adb-b9f3-b14f92ad7b09 None None] MFA Rules not processed for user `c3749568726644be9ebdf4d87463ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:13.336323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-68d9ff53-0bb7-4adb-b9f3-b14f92ad7b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:13.380712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:13.446791 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 519/1032] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:40:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:16.991664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6791ad-a62e-4cd3-840b-e66f144208ea None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:16.992751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6791ad-a62e-4cd3-840b-e66f144208ea None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:16.993032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6791ad-a62e-4cd3-840b-e66f144208ea None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:16.994468 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 514/1033] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:17.000480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff970-d487-411a-8c2f-977fef219cb4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:17.001007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff970-d487-411a-8c2f-977fef219cb4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:17.001007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff970-d487-411a-8c2f-977fef219cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:17.078777 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-693ff970-d487-411a-8c2f-977fef219cb4 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:17.079897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-693ff970-d487-411a-8c2f-977fef219cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:17.184940 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 520/1034] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:17.475514 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84d187-67f5-4e28-bc7f-f53fd6f8b4dd None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:17.476288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84d187-67f5-4e28-bc7f-f53fd6f8b4dd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:17.476892 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84d187-67f5-4e28-bc7f-f53fd6f8b4dd None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:17.479011 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 515/1035] 10.222.0.26 () {54 vars in 961 bytes} [Mon Oct 24 08:40:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:17.486571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbde71-79b3-452e-8e7e-e37e2f6456bc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:17.487508 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbde71-79b3-452e-8e7e-e37e2f6456bc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:17.488011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbde71-79b3-452e-8e7e-e37e2f6456bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:17.509852 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:17.583357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d1dbde71-79b3-452e-8e7e-e37e2f6456bc None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:17.584719 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d1dbde71-79b3-452e-8e7e-e37e2f6456bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:17.717464 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 521/1036] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Oct 24 08:40:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:40:17.728501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84d187-67f5-4e28-bc7f-f53fd6f8b4dd None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:17.731341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1d84d187-67f5-4e28-bc7f-f53fd6f8b4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:17.893797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84d187-67f5-4e28-bc7f-f53fd6f8b4dd None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:17.968761 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:18.014212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059f7ee-9dab-4534-bc29-a0e735f3f415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:18.015024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059f7ee-9dab-4534-bc29-a0e735f3f415 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:18.015411 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059f7ee-9dab-4534-bc29-a0e735f3f415 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:18.015788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059f7ee-9dab-4534-bc29-a0e735f3f415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:18.017524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2059f7ee-9dab-4534-bc29-a0e735f3f415 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:18.119097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059f7ee-9dab-4534-bc29-a0e735f3f415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:18.120874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059f7ee-9dab-4534-bc29-a0e735f3f415 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:18.140740 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 516/1037] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:40:20.481593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db4e4-a8db-4332-af7b-08f699116cdf None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:20.482196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db4e4-a8db-4332-af7b-08f699116cdf None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:20.482196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db4e4-a8db-4332-af7b-08f699116cdf None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:20.483503 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 522/1038] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:20.489401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514a286-9b7e-42fe-aa90-90e29389364f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:20.490101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514a286-9b7e-42fe-aa90-90e29389364f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:20.490101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514a286-9b7e-42fe-aa90-90e29389364f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:20.561072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d514a286-9b7e-42fe-aa90-90e29389364f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:20.562468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d514a286-9b7e-42fe-aa90-90e29389364f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:20.606386 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:20.678787 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 517/1039] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:20.968047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118db4e4-a8db-4332-af7b-08f699116cdf None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:20.969563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-118db4e4-a8db-4332-af7b-08f699116cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:21.043077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:21.150069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118db4e4-a8db-4332-af7b-08f699116cdf None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:21.280507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b271-5089-4fec-b4bc-c28971123e9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:21.281076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b271-5089-4fec-b4bc-c28971123e9d service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:21.281355 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b271-5089-4fec-b4bc-c28971123e9d service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:21.281627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b271-5089-4fec-b4bc-c28971123e9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:21.283497 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fa59b271-5089-4fec-b4bc-c28971123e9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:21.307878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:21.376428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b271-5089-4fec-b4bc-c28971123e9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:21.378281 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b271-5089-4fec-b4bc-c28971123e9d service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:21.396276 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 523/1040] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:23.854315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9892e0-7619-4838-9b45-9d747ed4c558 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:23.854914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9892e0-7619-4838-9b45-9d747ed4c558 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:23.854914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9892e0-7619-4838-9b45-9d747ed4c558 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:23.856338 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 518/1041] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:23.862027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e36f8-3864-47f9-9ff8-53c619ed4da2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:23.862468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e36f8-3864-47f9-9ff8-53c619ed4da2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:23.862641 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e36f8-3864-47f9-9ff8-53c619ed4da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:23.931093 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-eb0e36f8-3864-47f9-9ff8-53c619ed4da2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:23.932217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-eb0e36f8-3864-47f9-9ff8-53c619ed4da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:24.031150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:24.043064 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 524/1042] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:24.318671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb45ffe-90d6-48c0-aa70-8f29bbfe0d31 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:24.318985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb45ffe-90d6-48c0-aa70-8f29bbfe0d31 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:24.319260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb45ffe-90d6-48c0-aa70-8f29bbfe0d31 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:24.320732 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 519/1043] 10.222.0.26 () {54 vars in 961 bytes} [Mon Oct 24 08:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:24.326290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330299b2-8c64-41ae-aac5-b3e91d2c94b7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:24.326579 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330299b2-8c64-41ae-aac5-b3e91d2c94b7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:24.326884 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330299b2-8c64-41ae-aac5-b3e91d2c94b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:24.405853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-330299b2-8c64-41ae-aac5-b3e91d2c94b7 None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:24.407121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-330299b2-8c64-41ae-aac5-b3e91d2c94b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:24.531135 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 525/1044] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Oct 24 08:40:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:24.539539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb45ffe-90d6-48c0-aa70-8f29bbfe0d31 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:24.541009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9bb45ffe-90d6-48c0-aa70-8f29bbfe0d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:24.697384 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb45ffe-90d6-48c0-aa70-8f29bbfe0d31 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:24.718551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:24.822976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92c080-235f-48d2-b710-0ef388627c5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:24.823486 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92c080-235f-48d2-b710-0ef388627c5c service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:24.823764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92c080-235f-48d2-b710-0ef388627c5c service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:24.824029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92c080-235f-48d2-b710-0ef388627c5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:24.825778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6f92c080-235f-48d2-b710-0ef388627c5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:24.926408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92c080-235f-48d2-b710-0ef388627c5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:24.928093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92c080-235f-48d2-b710-0ef388627c5c service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:24.949893 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 520/1045] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:25.034072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853fc73-1ee2-487b-919a-594a8096608f None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:25.034664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853fc73-1ee2-487b-919a-594a8096608f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:25.034664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853fc73-1ee2-487b-919a-594a8096608f None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:25.036142 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 526/1046] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:25.041471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c21e12-cbd9-4298-988f-244ce994245e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:25.041800 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c21e12-cbd9-4298-988f-244ce994245e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:25.042456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c21e12-cbd9-4298-988f-244ce994245e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:25.078977 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:25.124875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-05c21e12-cbd9-4298-988f-244ce994245e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:25.126352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-05c21e12-cbd9-4298-988f-244ce994245e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:25.264770 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 521/1047] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:25.607572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a87a75-3293-45b3-9eb6-5dd32fdafe66 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:25.607979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a87a75-3293-45b3-9eb6-5dd32fdafe66 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:25.608363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a87a75-3293-45b3-9eb6-5dd32fdafe66 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:25.609963 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 527/1048] 10.222.0.26 () {54 vars in 963 bytes} [Mon Oct 24 08:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:25.615053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5e817-b99b-4b60-b355-032df18d0607 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:25.615444 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5e817-b99b-4b60-b355-032df18d0607 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:25.615825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5e817-b99b-4b60-b355-032df18d0607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:25.697182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6ca5e817-b99b-4b60-b355-032df18d0607 None None] MFA Rules not processed for user `6046d6440af84aa28aab47a018638e18`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:25.698550 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6ca5e817-b99b-4b60-b355-032df18d0607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:25.747040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:25.779285 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 522/1049] 10.222.0.26 () {58 vars in 1060 bytes} [Mon Oct 24 08:40:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:25.787409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a87a75-3293-45b3-9eb6-5dd32fdafe66 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:25.788921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-23a87a75-3293-45b3-9eb6-5dd32fdafe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:25.842506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:25.900736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a87a75-3293-45b3-9eb6-5dd32fdafe66 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:25.991314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7072e6-fbdf-4d36-a731-7e52f048c48d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:25.991947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7072e6-fbdf-4d36-a731-7e52f048c48d service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:25.992220 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7072e6-fbdf-4d36-a731-7e52f048c48d service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:25.992486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7072e6-fbdf-4d36-a731-7e52f048c48d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:25.994123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cd7072e6-fbdf-4d36-a731-7e52f048c48d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:26.118497 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:26.133570 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7072e6-fbdf-4d36-a731-7e52f048c48d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:26.135133 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7072e6-fbdf-4d36-a731-7e52f048c48d service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:26.156477 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 528/1050] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:27.220651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cde28-15db-4496-88a9-7ce2b22f2b87 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:27.221228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cde28-15db-4496-88a9-7ce2b22f2b87 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:27.221228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cde28-15db-4496-88a9-7ce2b22f2b87 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:27.222576 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 523/1051] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:27.227941 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b47c88-ac1e-4536-8a17-aef06cad609a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:27.228411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b47c88-ac1e-4536-8a17-aef06cad609a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:27.228489 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b47c88-ac1e-4536-8a17-aef06cad609a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:27.295254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-74b47c88-ac1e-4536-8a17-aef06cad609a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:27.296463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-74b47c88-ac1e-4536-8a17-aef06cad609a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:27.394683 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 529/1052] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:27.689550 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cde28-15db-4496-88a9-7ce2b22f2b87 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:27.691119 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-796cde28-15db-4496-88a9-7ce2b22f2b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:27.846091 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cde28-15db-4496-88a9-7ce2b22f2b87 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:27.900639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:27.963666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c5728-5fc8-4e58-b125-e7d3ded193e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:27.964283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c5728-5fc8-4e58-b125-e7d3ded193e1 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:27.964563 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c5728-5fc8-4e58-b125-e7d3ded193e1 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:27.964833 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c5728-5fc8-4e58-b125-e7d3ded193e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:27.966461 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-459c5728-5fc8-4e58-b125-e7d3ded193e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:28.060467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c5728-5fc8-4e58-b125-e7d3ded193e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:28.062086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c5728-5fc8-4e58-b125-e7d3ded193e1 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:28.080391 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 524/1053] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:30.281944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb789c9-4a98-423b-a211-b6ba41599355 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:30.281944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb789c9-4a98-423b-a211-b6ba41599355 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:30.281944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb789c9-4a98-423b-a211-b6ba41599355 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:30.286139 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 530/1054] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:30.287092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78396e8-e069-487b-83fe-c75542fca5aa None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:30.287502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78396e8-e069-487b-83fe-c75542fca5aa None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:30.287601 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78396e8-e069-487b-83fe-c75542fca5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:30.363939 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e78396e8-e069-487b-83fe-c75542fca5aa None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:30.365141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e78396e8-e069-487b-83fe-c75542fca5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:30.372899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:30.470526 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 525/1055] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:30.754764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb789c9-4a98-423b-a211-b6ba41599355 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:30.756419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ddb789c9-4a98-423b-a211-b6ba41599355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:30.817950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:30.905546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb789c9-4a98-423b-a211-b6ba41599355 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:31.023679 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbea4d-d19a-49e8-882b-ee6222d07a23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:31.024215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbea4d-d19a-49e8-882b-ee6222d07a23 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:31.024493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbea4d-d19a-49e8-882b-ee6222d07a23 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:31.024764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbea4d-d19a-49e8-882b-ee6222d07a23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:31.026338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-61cbea4d-d19a-49e8-882b-ee6222d07a23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:31.046267 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:31.120973 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbea4d-d19a-49e8-882b-ee6222d07a23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:31.122828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbea4d-d19a-49e8-882b-ee6222d07a23 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:31.141256 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 531/1056] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:33.349670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba7a54-b4ac-498e-b4f5-e4c7b4cfd128 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:33.350164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba7a54-b4ac-498e-b4f5-e4c7b4cfd128 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:33.350164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba7a54-b4ac-498e-b4f5-e4c7b4cfd128 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:33.351539 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 526/1057] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:33.357370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597761c-bc6a-4e04-b01d-6e222e72ec63 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:33.357801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597761c-bc6a-4e04-b01d-6e222e72ec63 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:33.357801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597761c-bc6a-4e04-b01d-6e222e72ec63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:33.482244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a597761c-bc6a-4e04-b01d-6e222e72ec63 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:33.484533 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a597761c-bc6a-4e04-b01d-6e222e72ec63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:33.583989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:33.590503 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 532/1058] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:33.959955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba7a54-b4ac-498e-b4f5-e4c7b4cfd128 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:33.961580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2dba7a54-b4ac-498e-b4f5-e4c7b4cfd128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:34.134863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:34.146626 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba7a54-b4ac-498e-b4f5-e4c7b4cfd128 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:34.282389 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480ffba-33da-4e50-a4b7-8f9de712d700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:34.283534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480ffba-33da-4e50-a4b7-8f9de712d700 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:34.283898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480ffba-33da-4e50-a4b7-8f9de712d700 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:34.284377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480ffba-33da-4e50-a4b7-8f9de712d700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:34.286453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e480ffba-33da-4e50-a4b7-8f9de712d700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:34.402988 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480ffba-33da-4e50-a4b7-8f9de712d700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:34.404936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480ffba-33da-4e50-a4b7-8f9de712d700 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:34.416851 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:34.427445 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 527/1059] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:36.654166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f681b6-5e7e-4c84-8601-a2c9a888f44e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:36.655266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f681b6-5e7e-4c84-8601-a2c9a888f44e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:36.655582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f681b6-5e7e-4c84-8601-a2c9a888f44e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:36.657438 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 533/1060] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:36.664873 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d14e76-1693-4b19-b0c4-f7a8d7d36e06 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:36.665285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d14e76-1693-4b19-b0c4-f7a8d7d36e06 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:36.665606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d14e76-1693-4b19-b0c4-f7a8d7d36e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:36.749379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-67d14e76-1693-4b19-b0c4-f7a8d7d36e06 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:36.750682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-67d14e76-1693-4b19-b0c4-f7a8d7d36e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:36.855305 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 528/1061] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:37.148986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f681b6-5e7e-4c84-8601-a2c9a888f44e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:37.150744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-24f681b6-5e7e-4c84-8601-a2c9a888f44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:37.302858 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f681b6-5e7e-4c84-8601-a2c9a888f44e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:37.375106 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:37.427845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fadfbe-0e01-4457-9b0b-e992853ee280 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:37.428410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fadfbe-0e01-4457-9b0b-e992853ee280 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:37.428689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fadfbe-0e01-4457-9b0b-e992853ee280 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:37.428963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fadfbe-0e01-4457-9b0b-e992853ee280 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:37.430932 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-95fadfbe-0e01-4457-9b0b-e992853ee280 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:37.546216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fadfbe-0e01-4457-9b0b-e992853ee280 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:37.547925 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fadfbe-0e01-4457-9b0b-e992853ee280 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:37.567753 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 534/1062] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:39.856757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ec654-d8b5-4f00-9cc1-931bc1241a3a None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:39.857304 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ec654-d8b5-4f00-9cc1-931bc1241a3a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:39.857304 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ec654-d8b5-4f00-9cc1-931bc1241a3a None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:39.858653 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 529/1063] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:39.864398 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138961f-9182-4acd-9b9c-0da9e7f8dbc5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:39.864884 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138961f-9182-4acd-9b9c-0da9e7f8dbc5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:39.864964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138961f-9182-4acd-9b9c-0da9e7f8dbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:39.932479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2138961f-9182-4acd-9b9c-0da9e7f8dbc5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:39.933763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2138961f-9182-4acd-9b9c-0da9e7f8dbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:39.969413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:40.034195 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 535/1064] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:40.317303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ec654-d8b5-4f00-9cc1-931bc1241a3a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:40.318865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-126ec654-d8b5-4f00-9cc1-931bc1241a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:40.397173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:40.481043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ec654-d8b5-4f00-9cc1-931bc1241a3a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:40.596671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da6e6b-131d-4156-a6cf-ee7f24d3907b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:40.597267 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da6e6b-131d-4156-a6cf-ee7f24d3907b service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:40.597531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da6e6b-131d-4156-a6cf-ee7f24d3907b service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:40.597795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da6e6b-131d-4156-a6cf-ee7f24d3907b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:40.599385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-77da6e6b-131d-4156-a6cf-ee7f24d3907b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:40.638853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:40.693338 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da6e6b-131d-4156-a6cf-ee7f24d3907b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:40.694953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da6e6b-131d-4156-a6cf-ee7f24d3907b service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:40.713405 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 530/1065] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:41.208336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7960c-e1e4-4036-851a-41a3b64aaf9e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:41.208942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7960c-e1e4-4036-851a-41a3b64aaf9e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:41.208942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7960c-e1e4-4036-851a-41a3b64aaf9e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:41.210260 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 536/1066] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:41.216035 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b02f-35f3-4cc4-a73d-ba94188442a5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:41.216464 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b02f-35f3-4cc4-a73d-ba94188442a5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:41.216569 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b02f-35f3-4cc4-a73d-ba94188442a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:41.290897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2197b02f-35f3-4cc4-a73d-ba94188442a5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:41.292142 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2197b02f-35f3-4cc4-a73d-ba94188442a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:41.393868 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 531/1067] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:41.687593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796a179-1019-4d30-9a17-b9e1d42e1465 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:41.687878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796a179-1019-4d30-9a17-b9e1d42e1465 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:41.688153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796a179-1019-4d30-9a17-b9e1d42e1465 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:41.689536 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 537/1068] 10.222.0.26 () {54 vars in 962 bytes} [Mon Oct 24 08:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:41.694910 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb0bde-64ac-472b-8112-a0a5855e1ef5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:41.695241 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb0bde-64ac-472b-8112-a0a5855e1ef5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:41.695514 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb0bde-64ac-472b-8112-a0a5855e1ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:41.711821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:41.774952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9aeb0bde-64ac-472b-8112-a0a5855e1ef5 None None] MFA Rules not processed for user `3b1c2ea68e4741728469ab7cc542b6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:41.776176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9aeb0bde-64ac-472b-8112-a0a5855e1ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:41.854345 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 532/1069] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Oct 24 08:40:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:41.862929 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796a179-1019-4d30-9a17-b9e1d42e1465 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:41.864459 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a796a179-1019-4d30-9a17-b9e1d42e1465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:41.969916 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796a179-1019-4d30-9a17-b9e1d42e1465 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:42.030224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:42.046431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037708f-318d-451b-b31a-b993e4bc88d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b1c2ea68e4741728469ab7cc542b6d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:42.047109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037708f-318d-451b-b31a-b993e4bc88d6 service glance] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:42.047490 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037708f-318d-451b-b31a-b993e4bc88d6 service glance] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:42.047764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037708f-318d-451b-b31a-b993e4bc88d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:42.049351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0037708f-318d-451b-b31a-b993e4bc88d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:42.158057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037708f-318d-451b-b31a-b993e4bc88d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:42.160132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037708f-318d-451b-b31a-b993e4bc88d6 service glance] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:42.186748 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 538/1070] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:42.899475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd8b6d-d7f6-452d-842e-33e55877ee98 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:42.900053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd8b6d-d7f6-452d-842e-33e55877ee98 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:42.900053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd8b6d-d7f6-452d-842e-33e55877ee98 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:42.901325 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 533/1071] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:42.906625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afe21b-fabc-436d-8fb2-80eb66cedfcc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:42.907086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afe21b-fabc-436d-8fb2-80eb66cedfcc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:42.907274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afe21b-fabc-436d-8fb2-80eb66cedfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:42.939152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d26c20-d1ef-4f27-9f0f-35be99d4e431 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:42.939500 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d26c20-d1ef-4f27-9f0f-35be99d4e431 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:42.939834 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d26c20-d1ef-4f27-9f0f-35be99d4e431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:42.975923 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-37afe21b-fabc-436d-8fb2-80eb66cedfcc None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:42.977122 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-37afe21b-fabc-436d-8fb2-80eb66cedfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:43.008425 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-09d26c20-d1ef-4f27-9f0f-35be99d4e431 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:43.009828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-09d26c20-d1ef-4f27-9f0f-35be99d4e431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:43.020219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:43.065222 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:43.079624 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 539/1072] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:43.098831 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 534/1073] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:40:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:40:43.146382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2fe02-9450-470d-8bdb-38509882f930 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:43.146740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2fe02-9450-470d-8bdb-38509882f930 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:43.147021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2fe02-9450-470d-8bdb-38509882f930 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:43.148687 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 540/1074] 10.222.0.26 () {54 vars in 943 bytes} [Mon Oct 24 08:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:43.155116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1694419-4c1d-4625-b525-8e53529f954f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:43.155467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1694419-4c1d-4625-b525-8e53529f954f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:43.155803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1694419-4c1d-4625-b525-8e53529f954f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:43.233890 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e1694419-4c1d-4625-b525-8e53529f954f None None] MFA Rules not processed for user `dbf288ceecac4fec8b2161c55a61a8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:43.235209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e1694419-4c1d-4625-b525-8e53529f954f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:43.343156 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 535/1075] 10.222.0.26 () {58 vars in 1040 bytes} [Mon Oct 24 08:40:43 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:43.352561 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2fe02-9450-470d-8bdb-38509882f930 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:43.354152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9de2fe02-9450-470d-8bdb-38509882f930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:43.371444 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1694419-4c1d-4625-b525-8e53529f954f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:43.373207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e1694419-4c1d-4625-b525-8e53529f954f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:43.405462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:43.474369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1694419-4c1d-4625-b525-8e53529f954f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:43.489831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2fe02-9450-470d-8bdb-38509882f930 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:43.527159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:43.587882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc018a1-f656-45c1-b241-ec732d6c057b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:43.588492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc018a1-f656-45c1-b241-ec732d6c057b service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:43.588766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc018a1-f656-45c1-b241-ec732d6c057b service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:43.589034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc018a1-f656-45c1-b241-ec732d6c057b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:43.590643 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dcc018a1-f656-45c1-b241-ec732d6c057b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:43.596242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e879170-44a0-4d81-b704-675110e6825c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:43.596855 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e879170-44a0-4d81-b704-675110e6825c service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:43.597133 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e879170-44a0-4d81-b704-675110e6825c service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:43.597407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e879170-44a0-4d81-b704-675110e6825c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:43.599084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9e879170-44a0-4d81-b704-675110e6825c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:43.635764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:43.684556 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc018a1-f656-45c1-b241-ec732d6c057b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:43.685019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e879170-44a0-4d81-b704-675110e6825c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:43.686140 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc018a1-f656-45c1-b241-ec732d6c057b service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:43.686645 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e879170-44a0-4d81-b704-675110e6825c service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:43.695128 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 541/1076] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:40:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:40:43.704212 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 536/1077] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:40:45.922806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba52b5a-a4f3-4b07-986a-3d37bd5828fd None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:45.923503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba52b5a-a4f3-4b07-986a-3d37bd5828fd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:45.923503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba52b5a-a4f3-4b07-986a-3d37bd5828fd None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:45.924846 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 542/1078] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:45.930085 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983a2ad-f0f7-4149-8e08-6f38ba56600e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:45.930511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983a2ad-f0f7-4149-8e08-6f38ba56600e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:45.930664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983a2ad-f0f7-4149-8e08-6f38ba56600e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:46.002173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1983a2ad-f0f7-4149-8e08-6f38ba56600e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:46.003425 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1983a2ad-f0f7-4149-8e08-6f38ba56600e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:46.112866 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 537/1079] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:46.398003 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba52b5a-a4f3-4b07-986a-3d37bd5828fd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:46.399748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aba52b5a-a4f3-4b07-986a-3d37bd5828fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:46.480890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:46.558123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba52b5a-a4f3-4b07-986a-3d37bd5828fd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:46.680258 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c62ab-c9ff-4757-b2f9-211c7a84af03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:46.680903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c62ab-c9ff-4757-b2f9-211c7a84af03 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:46.681178 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c62ab-c9ff-4757-b2f9-211c7a84af03 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:46.681442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c62ab-c9ff-4757-b2f9-211c7a84af03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:46.683066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a06c62ab-c9ff-4757-b2f9-211c7a84af03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:46.718130 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:46.781567 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c62ab-c9ff-4757-b2f9-211c7a84af03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:46.783143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c62ab-c9ff-4757-b2f9-211c7a84af03 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:46.802655 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 543/1080] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:49.017714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a353cdc-f457-45b3-8723-94d16133de0d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:49.018283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a353cdc-f457-45b3-8723-94d16133de0d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:49.018283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a353cdc-f457-45b3-8723-94d16133de0d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:49.019953 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 538/1081] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:49.025128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23418fe0-e528-4745-9125-6ec9323c3567 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:49.025600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23418fe0-e528-4745-9125-6ec9323c3567 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:49.025691 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23418fe0-e528-4745-9125-6ec9323c3567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:49.097986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-23418fe0-e528-4745-9125-6ec9323c3567 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:49.099158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-23418fe0-e528-4745-9125-6ec9323c3567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:49.208928 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 544/1082] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:49.497153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a353cdc-f457-45b3-8723-94d16133de0d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:49.498693 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7a353cdc-f457-45b3-8723-94d16133de0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:49.527565 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:49.721998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a353cdc-f457-45b3-8723-94d16133de0d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:49.819308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:49.847524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d31e0-691a-43a7-b9dc-4b9ace168133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:49.848495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d31e0-691a-43a7-b9dc-4b9ace168133 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:49.848868 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d31e0-691a-43a7-b9dc-4b9ace168133 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:49.849204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d31e0-691a-43a7-b9dc-4b9ace168133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:49.850980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f23d31e0-691a-43a7-b9dc-4b9ace168133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:49.953122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d31e0-691a-43a7-b9dc-4b9ace168133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:49.954755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d31e0-691a-43a7-b9dc-4b9ace168133 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:49.979512 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 539/1083] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:40:52.193743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb34af-5e5a-4fe8-9c06-3b85d3a148ea None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:52.194243 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb34af-5e5a-4fe8-9c06-3b85d3a148ea None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:52.194335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb34af-5e5a-4fe8-9c06-3b85d3a148ea None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:52.195843 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 545/1084] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:40:52.201093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d001fb-00ff-4a8e-aec6-c0cf34bdbfc2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:52.201537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d001fb-00ff-4a8e-aec6-c0cf34bdbfc2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:52.201650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d001fb-00ff-4a8e-aec6-c0cf34bdbfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:52.275524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-20d001fb-00ff-4a8e-aec6-c0cf34bdbfc2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:40:52.276881 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-20d001fb-00ff-4a8e-aec6-c0cf34bdbfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:52.335168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:52.383402 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 540/1085] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:40:52.666972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb34af-5e5a-4fe8-9c06-3b85d3a148ea None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:40:52.669300 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d9eb34af-5e5a-4fe8-9c06-3b85d3a148ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:52.677800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:52.828070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb34af-5e5a-4fe8-9c06-3b85d3a148ea None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:40:52.919372 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:40:52.954287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b2108-6518-4df2-9533-31111a6f5a38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:40:52.954936 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b2108-6518-4df2-9533-31111a6f5a38 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:40:52.958651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b2108-6518-4df2-9533-31111a6f5a38 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:40:52.958651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b2108-6518-4df2-9533-31111a6f5a38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:40:52.959053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-218b2108-6518-4df2-9533-31111a6f5a38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:40:53.056176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b2108-6518-4df2-9533-31111a6f5a38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:40:53.057806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b2108-6518-4df2-9533-31111a6f5a38 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:40:53.078398 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 546/1086] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:02.381333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=19 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:03.076010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=59 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:06.150842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f003b-dfb2-44f4-967c-9bdb94fbd9d5 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:06.151463 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f003b-dfb2-44f4-967c-9bdb94fbd9d5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:06.151463 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f003b-dfb2-44f4-967c-9bdb94fbd9d5 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:06.152712 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 541/1087] 10.222.0.26 () {54 vars in 918 bytes} [Mon Oct 24 08:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:06.158374 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b1647-8bc6-4590-bbdb-dbd3bdbd26c6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:06.158857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b1647-8bc6-4590-bbdb-dbd3bdbd26c6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:06.158969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b1647-8bc6-4590-bbdb-dbd3bdbd26c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:06.241976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b18b1647-8bc6-4590-bbdb-dbd3bdbd26c6 None None] MFA Rules not processed for user `c3749568726644be9ebdf4d87463ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:06.243793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b18b1647-8bc6-4590-bbdb-dbd3bdbd26c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:06.353932 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 547/1088] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Oct 24 08:41:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:06.368778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e3324-3d65-45da-a941-4dae78309067 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:06.369063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e3324-3d65-45da-a941-4dae78309067 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:06.369336 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e3324-3d65-45da-a941-4dae78309067 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:06.370698 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 542/1089] 10.222.0.26 () {54 vars in 918 bytes} [Mon Oct 24 08:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:06.375670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fc436-cf63-4c8f-9635-5c8115888016 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:06.375955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fc436-cf63-4c8f-9635-5c8115888016 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:06.376229 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fc436-cf63-4c8f-9635-5c8115888016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:06.454792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:06.458067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6b7fc436-cf63-4c8f-9635-5c8115888016 None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:06.460150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6b7fc436-cf63-4c8f-9635-5c8115888016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:06.585689 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 548/1090] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Oct 24 08:41:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:09.159048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c14ae9-99c2-414e-923c-cde7aaa2896c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:09.159623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c14ae9-99c2-414e-923c-cde7aaa2896c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:09.159623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c14ae9-99c2-414e-923c-cde7aaa2896c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:09.160938 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 543/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:09.167298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92362769-18f9-40f1-8f13-41b1738ff2fc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:09.168080 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92362769-18f9-40f1-8f13-41b1738ff2fc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:09.168602 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92362769-18f9-40f1-8f13-41b1738ff2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:09.238218 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-92362769-18f9-40f1-8f13-41b1738ff2fc None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:09.239454 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-92362769-18f9-40f1-8f13-41b1738ff2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:09.296864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:09.340556 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 549/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:09.635536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c14ae9-99c2-414e-923c-cde7aaa2896c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:09.638074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b3c14ae9-99c2-414e-923c-cde7aaa2896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:09.802092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c14ae9-99c2-414e-923c-cde7aaa2896c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:09.883945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:09.928827 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e94df2-7cc4-4cbe-8e20-02bf5b446220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:09.929434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e94df2-7cc4-4cbe-8e20-02bf5b446220 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:09.929707 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e94df2-7cc4-4cbe-8e20-02bf5b446220 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:09.929978 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e94df2-7cc4-4cbe-8e20-02bf5b446220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:09.931593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-92e94df2-7cc4-4cbe-8e20-02bf5b446220 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:10.026253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e94df2-7cc4-4cbe-8e20-02bf5b446220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:10.027853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e94df2-7cc4-4cbe-8e20-02bf5b446220 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:10.046372 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 544/1093] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:11.174475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aae300-1928-4889-b3c8-947619f78cf3 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:11.175091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aae300-1928-4889-b3c8-947619f78cf3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:11.175091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aae300-1928-4889-b3c8-947619f78cf3 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:11.177173 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 550/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:41:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:11.185212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b170c7-2d0b-4dd2-8a41-10f4825e7604 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:11.185639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b170c7-2d0b-4dd2-8a41-10f4825e7604 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:11.185767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b170c7-2d0b-4dd2-8a41-10f4825e7604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:11.270374 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-74b170c7-2d0b-4dd2-8a41-10f4825e7604 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:11.271639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-74b170c7-2d0b-4dd2-8a41-10f4825e7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:11.301695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:11.382803 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 545/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:11.538515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aae300-1928-4889-b3c8-947619f78cf3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:11.539984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d9aae300-1928-4889-b3c8-947619f78cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:11.687416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aae300-1928-4889-b3c8-947619f78cf3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:11.727560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:11.806075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732756df-1bcf-4428-9de0-9d15066d6662 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:11.806593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732756df-1bcf-4428-9de0-9d15066d6662 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:11.806876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732756df-1bcf-4428-9de0-9d15066d6662 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:11.807198 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732756df-1bcf-4428-9de0-9d15066d6662 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:11.808734 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-732756df-1bcf-4428-9de0-9d15066d6662 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:11.905608 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732756df-1bcf-4428-9de0-9d15066d6662 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:11.907235 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732756df-1bcf-4428-9de0-9d15066d6662 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:11.926064 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 551/1096] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:13.039550 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588394e-0607-41bb-95f6-6be311ed2e70 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:13.040058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588394e-0607-41bb-95f6-6be311ed2e70 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:13.040058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588394e-0607-41bb-95f6-6be311ed2e70 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:13.041447 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 546/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:13.047738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1ce37-b4b0-4142-9c21-17d9a9437d12 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:13.048183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1ce37-b4b0-4142-9c21-17d9a9437d12 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:13.048306 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1ce37-b4b0-4142-9c21-17d9a9437d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:13.107998 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:13.131263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8ad1ce37-b4b0-4142-9c21-17d9a9437d12 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:13.132433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8ad1ce37-b4b0-4142-9c21-17d9a9437d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:13.232486 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 552/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:13.388324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588394e-0607-41bb-95f6-6be311ed2e70 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:13.390427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b588394e-0607-41bb-95f6-6be311ed2e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:13.569893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588394e-0607-41bb-95f6-6be311ed2e70 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:13.582073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:13.692582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1342866-02e1-4cfa-8ed5-903fad4ae537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:13.693075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1342866-02e1-4cfa-8ed5-903fad4ae537 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:13.693356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1342866-02e1-4cfa-8ed5-903fad4ae537 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:13.693624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1342866-02e1-4cfa-8ed5-903fad4ae537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:13.695234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e1342866-02e1-4cfa-8ed5-903fad4ae537 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:13.791400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1342866-02e1-4cfa-8ed5-903fad4ae537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:13.792983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1342866-02e1-4cfa-8ed5-903fad4ae537 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:13.811453 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 547/1099] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:14.902902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd08272-6cdc-4988-9485-64b77df271a6 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:14.903881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd08272-6cdc-4988-9485-64b77df271a6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:14.903881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd08272-6cdc-4988-9485-64b77df271a6 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:14.905922 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 553/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:41:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:14.913997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642ef09-c3a1-456f-b4ea-6b106f23b75f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:14.914755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642ef09-c3a1-456f-b4ea-6b106f23b75f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:14.914917 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642ef09-c3a1-456f-b4ea-6b106f23b75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:14.937714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:14.998656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c642ef09-c3a1-456f-b4ea-6b106f23b75f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:14.999887 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c642ef09-c3a1-456f-b4ea-6b106f23b75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:15.102372 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 548/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:15.251433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd08272-6cdc-4988-9485-64b77df271a6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:15.252835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8bd08272-6cdc-4988-9485-64b77df271a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:15.372040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:15.409222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd08272-6cdc-4988-9485-64b77df271a6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:15.536039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea6aa2-1752-4340-b10d-e18160ca2871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:15.536580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea6aa2-1752-4340-b10d-e18160ca2871 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:15.536863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea6aa2-1752-4340-b10d-e18160ca2871 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:15.537138 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea6aa2-1752-4340-b10d-e18160ca2871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:15.538765 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f4ea6aa2-1752-4340-b10d-e18160ca2871 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:15.617361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:15.638125 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea6aa2-1752-4340-b10d-e18160ca2871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:15.639762 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea6aa2-1752-4340-b10d-e18160ca2871 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:15.658054 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 554/1102] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:16.772176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2e33e-7a19-4cde-b2ef-6cbda9fd24a8 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:16.773181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2e33e-7a19-4cde-b2ef-6cbda9fd24a8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:16.773181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2e33e-7a19-4cde-b2ef-6cbda9fd24a8 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:16.775167 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 549/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:41:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:16.780264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173c97a-fd12-43cc-ad83-08ae61c8b8a7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:16.780746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173c97a-fd12-43cc-ad83-08ae61c8b8a7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:16.780746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173c97a-fd12-43cc-ad83-08ae61c8b8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:16.865496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0173c97a-fd12-43cc-ad83-08ae61c8b8a7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:16.866731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0173c97a-fd12-43cc-ad83-08ae61c8b8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:16.964038 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 555/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:17.112562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2e33e-7a19-4cde-b2ef-6cbda9fd24a8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:17.114068 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-00e2e33e-7a19-4cde-b2ef-6cbda9fd24a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:17.201671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:17.272650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2e33e-7a19-4cde-b2ef-6cbda9fd24a8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:17.399553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f58749-9d1e-4e08-b92c-6b210c69b864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:17.400163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f58749-9d1e-4e08-b92c-6b210c69b864 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:17.400441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f58749-9d1e-4e08-b92c-6b210c69b864 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:17.400710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f58749-9d1e-4e08-b92c-6b210c69b864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:17.402306 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b3f58749-9d1e-4e08-b92c-6b210c69b864 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:17.453431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:17.497542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f58749-9d1e-4e08-b92c-6b210c69b864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:17.499191 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f58749-9d1e-4e08-b92c-6b210c69b864 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:17.525740 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 550/1105] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:18.568601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2703fb5-99d5-4f3c-8690-e57c8c7a28e1 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:18.569143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2703fb5-99d5-4f3c-8690-e57c8c7a28e1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:18.569143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2703fb5-99d5-4f3c-8690-e57c8c7a28e1 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:18.570476 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 556/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:18.575537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854cf64-c5eb-48df-8cc4-6b6f63647b43 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:18.575978 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854cf64-c5eb-48df-8cc4-6b6f63647b43 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:18.576121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854cf64-c5eb-48df-8cc4-6b6f63647b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:18.661406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b854cf64-c5eb-48df-8cc4-6b6f63647b43 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:18.662635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b854cf64-c5eb-48df-8cc4-6b6f63647b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:18.765584 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 551/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:18.919504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2703fb5-99d5-4f3c-8690-e57c8c7a28e1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:18.921030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d2703fb5-99d5-4f3c-8690-e57c8c7a28e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:18.972127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:19.072417 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2703fb5-99d5-4f3c-8690-e57c8c7a28e1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:19.187547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158cb49-b0a8-4078-a5dc-0195fef8540a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:19.188047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158cb49-b0a8-4078-a5dc-0195fef8540a service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:19.188313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158cb49-b0a8-4078-a5dc-0195fef8540a service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:19.188584 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158cb49-b0a8-4078-a5dc-0195fef8540a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:19.190164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c158cb49-b0a8-4078-a5dc-0195fef8540a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:19.197077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:19.282313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158cb49-b0a8-4078-a5dc-0195fef8540a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:19.284554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158cb49-b0a8-4078-a5dc-0195fef8540a service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:19.303597 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 557/1108] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:20.357882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd77970-3b56-44b5-8c3c-2b3c6e9abb62 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:20.358452 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd77970-3b56-44b5-8c3c-2b3c6e9abb62 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:20.358452 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd77970-3b56-44b5-8c3c-2b3c6e9abb62 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:20.359875 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 552/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:20.367635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35ca33-7679-495d-815d-3c00adf4d118 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:20.368083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35ca33-7679-495d-815d-3c00adf4d118 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:20.368184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35ca33-7679-495d-815d-3c00adf4d118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:20.458505 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8d35ca33-7679-495d-815d-3c00adf4d118 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:20.459714 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8d35ca33-7679-495d-815d-3c00adf4d118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:20.536952 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:20.567789 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 558/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:20.719727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd77970-3b56-44b5-8c3c-2b3c6e9abb62 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:20.721546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0fd77970-3b56-44b5-8c3c-2b3c6e9abb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:20.748455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:20.889557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd77970-3b56-44b5-8c3c-2b3c6e9abb62 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:20.982313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:21.010007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe6ac2-054e-4901-87e2-d0f5c52c2212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:21.010617 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe6ac2-054e-4901-87e2-d0f5c52c2212 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:21.010936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe6ac2-054e-4901-87e2-d0f5c52c2212 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:21.011240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe6ac2-054e-4901-87e2-d0f5c52c2212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:21.012948 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-49fe6ac2-054e-4901-87e2-d0f5c52c2212 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:21.105605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe6ac2-054e-4901-87e2-d0f5c52c2212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:21.107215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe6ac2-054e-4901-87e2-d0f5c52c2212 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:21.125518 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 553/1111] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:22.158656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed6386-cc6b-4492-b3ca-98a577878415 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:22.159148 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed6386-cc6b-4492-b3ca-98a577878415 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:22.160266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed6386-cc6b-4492-b3ca-98a577878415 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:22.162491 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 559/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:41:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:22.169947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766044b5-c72b-4c98-8924-5bb04986b9ad None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:22.170452 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766044b5-c72b-4c98-8924-5bb04986b9ad None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:22.170745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766044b5-c72b-4c98-8924-5bb04986b9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:22.259981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-766044b5-c72b-4c98-8924-5bb04986b9ad None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:22.261228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-766044b5-c72b-4c98-8924-5bb04986b9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:22.309373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:22.364731 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 554/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:41:22.516393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed6386-cc6b-4492-b3ca-98a577878415 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:22.517884 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c3ed6386-cc6b-4492-b3ca-98a577878415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:22.681106 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed6386-cc6b-4492-b3ca-98a577878415 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:22.735944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:22.797186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49677e-4eed-48de-a67e-d18f71e682b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:22.797725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49677e-4eed-48de-a67e-d18f71e682b1 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:22.798002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49677e-4eed-48de-a67e-d18f71e682b1 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:22.798269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49677e-4eed-48de-a67e-d18f71e682b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:22.799895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cc49677e-4eed-48de-a67e-d18f71e682b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:22.898565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49677e-4eed-48de-a67e-d18f71e682b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:22.900964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49677e-4eed-48de-a67e-d18f71e682b1 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:22.921742 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 560/1114] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:23.953318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378048eb-d252-4165-915f-f5d573e3450f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:23.954238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378048eb-d252-4165-915f-f5d573e3450f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:23.954238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378048eb-d252-4165-915f-f5d573e3450f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:23.956256 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 555/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:41:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:23.962384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0542c-1170-48fe-97bb-71669a81243f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:23.962837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0542c-1170-48fe-97bb-71669a81243f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:23.962980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0542c-1170-48fe-97bb-71669a81243f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:24.048037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-eaf0542c-1170-48fe-97bb-71669a81243f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:24.049256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-eaf0542c-1170-48fe-97bb-71669a81243f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:24.051425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:24.152870 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 561/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:24.304361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378048eb-d252-4165-915f-f5d573e3450f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:24.305863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-378048eb-d252-4165-915f-f5d573e3450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:24.461235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378048eb-d252-4165-915f-f5d573e3450f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:24.492005 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:24.581681 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1725a70-cd6b-4ae6-9fc5-951ad1336795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:24.582286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1725a70-cd6b-4ae6-9fc5-951ad1336795 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:24.582563 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1725a70-cd6b-4ae6-9fc5-951ad1336795 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:24.582861 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1725a70-cd6b-4ae6-9fc5-951ad1336795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:24.584440 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b1725a70-cd6b-4ae6-9fc5-951ad1336795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:24.680453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1725a70-cd6b-4ae6-9fc5-951ad1336795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:24.682028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1725a70-cd6b-4ae6-9fc5-951ad1336795 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:24.701073 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 556/1117] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:26.754169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ef833-1d4c-4f1a-b11a-323dad5ec04d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:26.754800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ef833-1d4c-4f1a-b11a-323dad5ec04d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:26.754800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ef833-1d4c-4f1a-b11a-323dad5ec04d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:26.756857 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 562/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:26.765677 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6cf9b-a233-4c86-9488-fdca7dea11dc None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:26.766096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6cf9b-a233-4c86-9488-fdca7dea11dc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:26.766262 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6cf9b-a233-4c86-9488-fdca7dea11dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:26.815581 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:26.842870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d1f6cf9b-a233-4c86-9488-fdca7dea11dc None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:26.844096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d1f6cf9b-a233-4c86-9488-fdca7dea11dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:26.952727 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 557/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:27.245910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ef833-1d4c-4f1a-b11a-323dad5ec04d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:27.248379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-114ef833-1d4c-4f1a-b11a-323dad5ec04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:27.361328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ef833-1d4c-4f1a-b11a-323dad5ec04d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:27.394480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:27.441810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062f8e6-67f3-4da3-ba28-444f254037f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:27.442449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062f8e6-67f3-4da3-ba28-444f254037f5 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:27.442868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062f8e6-67f3-4da3-ba28-444f254037f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:27.443100 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062f8e6-67f3-4da3-ba28-444f254037f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:27.444889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1062f8e6-67f3-4da3-ba28-444f254037f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:27.568712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062f8e6-67f3-4da3-ba28-444f254037f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:27.570062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062f8e6-67f3-4da3-ba28-444f254037f5 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:27.589551 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 563/1120] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:29.880848 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e41e0d-c348-47ef-a3d3-4e077a56183b None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:29.881734 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e41e0d-c348-47ef-a3d3-4e077a56183b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:29.881734 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e41e0d-c348-47ef-a3d3-4e077a56183b None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:29.883840 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 558/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:29.891665 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6edda1-b285-4c3f-a148-4b48a53291d9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:29.892077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6edda1-b285-4c3f-a148-4b48a53291d9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:29.892215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6edda1-b285-4c3f-a148-4b48a53291d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:29.961591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0e6edda1-b285-4c3f-a148-4b48a53291d9 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:29.963409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0e6edda1-b285-4c3f-a148-4b48a53291d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:29.983229 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:30.072481 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 564/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:30.354795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e41e0d-c348-47ef-a3d3-4e077a56183b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:30.357842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-51e41e0d-c348-47ef-a3d3-4e077a56183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:30.471146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e41e0d-c348-47ef-a3d3-4e077a56183b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:30.484436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:30.551414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d71591-cd0a-4c8e-a9b1-4807493d7af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:30.552043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d71591-cd0a-4c8e-a9b1-4807493d7af7 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:30.552314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d71591-cd0a-4c8e-a9b1-4807493d7af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:30.552591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d71591-cd0a-4c8e-a9b1-4807493d7af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:30.554158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-73d71591-cd0a-4c8e-a9b1-4807493d7af7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:30.665908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d71591-cd0a-4c8e-a9b1-4807493d7af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:30.667388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d71591-cd0a-4c8e-a9b1-4807493d7af7 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:30.687247 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 559/1123] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:33.150059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935a62a-183a-403d-90a9-0b699a72070e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:33.150625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935a62a-183a-403d-90a9-0b699a72070e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:33.150625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935a62a-183a-403d-90a9-0b699a72070e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:33.152067 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 565/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:33.158528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca828d7a-994e-4ad5-9dca-788c9f5dd297 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:33.159215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca828d7a-994e-4ad5-9dca-788c9f5dd297 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:33.159505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca828d7a-994e-4ad5-9dca-788c9f5dd297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:33.215995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:33.236020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ca828d7a-994e-4ad5-9dca-788c9f5dd297 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:33.237228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ca828d7a-994e-4ad5-9dca-788c9f5dd297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:33.336296 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 560/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:33.622651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935a62a-183a-403d-90a9-0b699a72070e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:33.624287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5935a62a-183a-403d-90a9-0b699a72070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:33.735985 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935a62a-183a-403d-90a9-0b699a72070e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:33.775947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:33.810605 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd7a93-f747-4e4c-906b-ea08a93673aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:33.811259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd7a93-f747-4e4c-906b-ea08a93673aa service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:33.811534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd7a93-f747-4e4c-906b-ea08a93673aa service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:33.811865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd7a93-f747-4e4c-906b-ea08a93673aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:33.813451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ebfd7a93-f747-4e4c-906b-ea08a93673aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:33.927010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd7a93-f747-4e4c-906b-ea08a93673aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:33.928357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd7a93-f747-4e4c-906b-ea08a93673aa service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:33.947365 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 566/1126] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:36.211875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8cfce-a0e5-43cd-bad1-17a868182342 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:36.212819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8cfce-a0e5-43cd-bad1-17a868182342 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:36.212819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8cfce-a0e5-43cd-bad1-17a868182342 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:36.214622 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 561/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:36.221229 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bead2f8-7cd1-479b-aa69-8d4444f57193 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:36.221631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bead2f8-7cd1-479b-aa69-8d4444f57193 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:36.221774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bead2f8-7cd1-479b-aa69-8d4444f57193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:36.290845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3bead2f8-7cd1-479b-aa69-8d4444f57193 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:36.292041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3bead2f8-7cd1-479b-aa69-8d4444f57193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:36.314103 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:36.393859 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 567/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:36.682236 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8cfce-a0e5-43cd-bad1-17a868182342 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:36.683884 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5cb8cfce-a0e5-43cd-bad1-17a868182342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:36.794461 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8cfce-a0e5-43cd-bad1-17a868182342 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:36.816037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:36.871488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dd413-0d47-4646-b3f3-3df286333d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:36.872095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd413-0d47-4646-b3f3-3df286333d98 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:36.872399 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd413-0d47-4646-b3f3-3df286333d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:36.872616 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dd413-0d47-4646-b3f3-3df286333d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:36.874278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-971dd413-0d47-4646-b3f3-3df286333d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:36.989681 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dd413-0d47-4646-b3f3-3df286333d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:36.991067 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dd413-0d47-4646-b3f3-3df286333d98 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:37.008484 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 562/1129] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:39.264370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715b6aa-7746-46c3-a0a5-f0870c93f7da None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:39.265340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715b6aa-7746-46c3-a0a5-f0870c93f7da None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:39.265340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715b6aa-7746-46c3-a0a5-f0870c93f7da None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:39.266816 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 568/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:39.272340 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671535f-a1d7-42eb-97e5-2fda0157a324 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:39.272792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671535f-a1d7-42eb-97e5-2fda0157a324 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:39.272885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671535f-a1d7-42eb-97e5-2fda0157a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:39.339201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:39.342013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d671535f-a1d7-42eb-97e5-2fda0157a324 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:39.343740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d671535f-a1d7-42eb-97e5-2fda0157a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:39.443061 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 563/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:41:39.736135 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715b6aa-7746-46c3-a0a5-f0870c93f7da None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:39.737691 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3715b6aa-7746-46c3-a0a5-f0870c93f7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:39.847493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715b6aa-7746-46c3-a0a5-f0870c93f7da None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:39.898158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:39.923373 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de0fc-7056-4d86-936f-e44182fcb2f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:39.923984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de0fc-7056-4d86-936f-e44182fcb2f0 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:39.924261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de0fc-7056-4d86-936f-e44182fcb2f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:39.924528 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de0fc-7056-4d86-936f-e44182fcb2f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:39.926112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9d6de0fc-7056-4d86-936f-e44182fcb2f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:40.038580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6de0fc-7056-4d86-936f-e44182fcb2f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:40.039980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6de0fc-7056-4d86-936f-e44182fcb2f0 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:40.058286 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 569/1132] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:42.230337 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd17f6-ea20-470c-ad8b-79aa75a1575b None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:42.230843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd17f6-ea20-470c-ad8b-79aa75a1575b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:42.230939 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd17f6-ea20-470c-ad8b-79aa75a1575b None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:42.232252 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 564/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:42.238956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e07dae-a723-40a4-99b2-c71ba2c7c34c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:42.239389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e07dae-a723-40a4-99b2-c71ba2c7c34c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:42.239555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e07dae-a723-40a4-99b2-c71ba2c7c34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:42.313949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-77e07dae-a723-40a4-99b2-c71ba2c7c34c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:42.315181 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-77e07dae-a723-40a4-99b2-c71ba2c7c34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:42.344210 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:42.412182 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 570/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:42.692290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd17f6-ea20-470c-ad8b-79aa75a1575b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:42.693813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3dfd17f6-ea20-470c-ad8b-79aa75a1575b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:42.799305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd17f6-ea20-470c-ad8b-79aa75a1575b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:42.828221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:42.877744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50aa77-b7e4-4a47-b6eb-7012874e958a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:42.878565 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50aa77-b7e4-4a47-b6eb-7012874e958a service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:42.878853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50aa77-b7e4-4a47-b6eb-7012874e958a service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:42.879211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50aa77-b7e4-4a47-b6eb-7012874e958a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:42.881733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-af50aa77-b7e4-4a47-b6eb-7012874e958a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:43.002030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50aa77-b7e4-4a47-b6eb-7012874e958a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:43.003430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50aa77-b7e4-4a47-b6eb-7012874e958a service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:43.022874 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 565/1135] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:41:45.219741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d8a11-144f-4328-b9c1-d5135a939bc5 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:45.220721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d8a11-144f-4328-b9c1-d5135a939bc5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:45.220721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d8a11-144f-4328-b9c1-d5135a939bc5 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:45.222773 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 571/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:45.231157 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dc2ff-074e-4851-9455-b35ee736c47c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:45.231626 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dc2ff-074e-4851-9455-b35ee736c47c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:45.231728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dc2ff-074e-4851-9455-b35ee736c47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:45.304507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cc8dc2ff-074e-4851-9455-b35ee736c47c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:45.306228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cc8dc2ff-074e-4851-9455-b35ee736c47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:45.316520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:45.413648 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 566/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:41:45.699463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d8a11-144f-4328-b9c1-d5135a939bc5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:45.702005 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dc7d8a11-144f-4328-b9c1-d5135a939bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:45.814384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d8a11-144f-4328-b9c1-d5135a939bc5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:45.874256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:45.893720 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4567f-a892-4032-8128-1fba82246124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:45.894337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4567f-a892-4032-8128-1fba82246124 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:45.894612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4567f-a892-4032-8128-1fba82246124 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:45.894910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4567f-a892-4032-8128-1fba82246124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:45.896493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d5d4567f-a892-4032-8128-1fba82246124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:46.008598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4567f-a892-4032-8128-1fba82246124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:46.009949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4567f-a892-4032-8128-1fba82246124 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:46.032312 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 572/1138] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:48.225753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c79a7-8514-40a3-991e-1f25292e05c5 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:48.226472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c79a7-8514-40a3-991e-1f25292e05c5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:48.226866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c79a7-8514-40a3-991e-1f25292e05c5 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:48.228745 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 567/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:48.237486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2d5fb-d972-420d-83e7-01264257d3ff None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:48.238262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2d5fb-d972-420d-83e7-01264257d3ff None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:48.238433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2d5fb-d972-420d-83e7-01264257d3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:48.311179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5cd2d5fb-d972-420d-83e7-01264257d3ff None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:48.312390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5cd2d5fb-d972-420d-83e7-01264257d3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:48.352572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:48.416919 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 573/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:41:48.696802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c79a7-8514-40a3-991e-1f25292e05c5 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:48.698389 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-197c79a7-8514-40a3-991e-1f25292e05c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:48.803041 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c79a7-8514-40a3-991e-1f25292e05c5 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:48.841949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:48.880924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a36a08-5bef-4202-8483-a8008cb72442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:48.881590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a36a08-5bef-4202-8483-a8008cb72442 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:48.881874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a36a08-5bef-4202-8483-a8008cb72442 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:48.882146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a36a08-5bef-4202-8483-a8008cb72442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:48.883787 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-48a36a08-5bef-4202-8483-a8008cb72442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:48.998998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a36a08-5bef-4202-8483-a8008cb72442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:49.000425 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a36a08-5bef-4202-8483-a8008cb72442 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:49.018537 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 568/1141] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:51.168243 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055914f-26d2-49c6-8f86-cbbbdf6476ed None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:51.168847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055914f-26d2-49c6-8f86-cbbbdf6476ed None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:51.168847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055914f-26d2-49c6-8f86-cbbbdf6476ed None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:51.170579 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 574/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:51.176370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee61d1-51f8-4dcd-a7a6-da703b1477af None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:51.176831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee61d1-51f8-4dcd-a7a6-da703b1477af None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:51.176938 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee61d1-51f8-4dcd-a7a6-da703b1477af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:51.244650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-43ee61d1-51f8-4dcd-a7a6-da703b1477af None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:51.245848 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-43ee61d1-51f8-4dcd-a7a6-da703b1477af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:51.265132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:51.348468 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 569/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:41:51.629935 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055914f-26d2-49c6-8f86-cbbbdf6476ed None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:51.632416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a055914f-26d2-49c6-8f86-cbbbdf6476ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:51.746568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055914f-26d2-49c6-8f86-cbbbdf6476ed None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:51.818115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:51.825264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b25a78-4dac-4c71-b558-007118efdaf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:51.825875 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b25a78-4dac-4c71-b558-007118efdaf2 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:51.826160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b25a78-4dac-4c71-b558-007118efdaf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:51.826431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b25a78-4dac-4c71-b558-007118efdaf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:51.828102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-44b25a78-4dac-4c71-b558-007118efdaf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:51.942585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b25a78-4dac-4c71-b558-007118efdaf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:51.943995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b25a78-4dac-4c71-b558-007118efdaf2 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:51.962947 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 575/1144] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:41:54.171106 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa53c8-4665-4622-b98f-61f578e841f5 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:54.171712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa53c8-4665-4622-b98f-61f578e841f5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:54.171712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa53c8-4665-4622-b98f-61f578e841f5 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:54.173011 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 570/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:41:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:41:54.178743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4da323-43e3-4b78-b4b1-011c30238ed6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:54.179208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4da323-43e3-4b78-b4b1-011c30238ed6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:54.179302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4da323-43e3-4b78-b4b1-011c30238ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:54.245457 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3a4da323-43e3-4b78-b4b1-011c30238ed6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:41:54.246658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3a4da323-43e3-4b78-b4b1-011c30238ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:54.295948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:54.345277 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 576/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:41:54.631201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa53c8-4665-4622-b98f-61f578e841f5 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:41:54.633649 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-50fa53c8-4665-4622-b98f-61f578e841f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:54.741232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa53c8-4665-4622-b98f-61f578e841f5 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:41:54.789367 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:41:54.817849 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82819f7-62f4-43ec-b11f-21c26b77efd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:41:54.818488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82819f7-62f4-43ec-b11f-21c26b77efd3 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:41:54.818760 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82819f7-62f4-43ec-b11f-21c26b77efd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:41:54.819059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82819f7-62f4-43ec-b11f-21c26b77efd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:41:54.820640 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c82819f7-62f4-43ec-b11f-21c26b77efd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:41:54.941177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82819f7-62f4-43ec-b11f-21c26b77efd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:41:54.942561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82819f7-62f4-43ec-b11f-21c26b77efd3 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:41:54.968008 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 571/1147] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:42:04.343718 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=22 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:04.964454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=66 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:14.642298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00b4e7-1162-45fe-b3ff-d24667dc0460 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:14.642904 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00b4e7-1162-45fe-b3ff-d24667dc0460 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:14.642904 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00b4e7-1162-45fe-b3ff-d24667dc0460 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:14.644230 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 577/1148] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:42:14.650549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53db6-7129-42db-a824-65daef8fa412 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:14.651058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53db6-7129-42db-a824-65daef8fa412 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:14.651163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53db6-7129-42db-a824-65daef8fa412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:14.723637 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-34a53db6-7129-42db-a824-65daef8fa412 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:42:14.724846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-34a53db6-7129-42db-a824-65daef8fa412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:14.832256 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 572/1149] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:42:15.142730 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c70a6-8b73-495e-99e5-a3bea31e2e35 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:15.143029 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c70a6-8b73-495e-99e5-a3bea31e2e35 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:15.143302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c70a6-8b73-495e-99e5-a3bea31e2e35 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:15.144637 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 578/1150] 10.222.0.26 () {54 vars in 962 bytes} [Mon Oct 24 08:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:42:15.149700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b85e5-860b-4514-9190-c3e0ba8faf22 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:15.149983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b85e5-860b-4514-9190-c3e0ba8faf22 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:15.150261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b85e5-860b-4514-9190-c3e0ba8faf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:15.231232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8e3b85e5-860b-4514-9190-c3e0ba8faf22 None None] MFA Rules not processed for user `3b1c2ea68e4741728469ab7cc542b6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:42:15.232458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8e3b85e5-860b-4514-9190-c3e0ba8faf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:15.255536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:15.314361 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 573/1151] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Oct 24 08:42:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:42:15.322511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c70a6-8b73-495e-99e5-a3bea31e2e35 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:42:15.324066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-832c70a6-8b73-495e-99e5-a3bea31e2e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:15.436773 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c70a6-8b73-495e-99e5-a3bea31e2e35 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:42:15.514385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d497dd-451d-4160-930e-af04bf947cc6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b1c2ea68e4741728469ab7cc542b6d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:42:15.515032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d497dd-451d-4160-930e-af04bf947cc6 service glance] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:15.515311 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d497dd-451d-4160-930e-af04bf947cc6 service glance] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:15.515581 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d497dd-451d-4160-930e-af04bf947cc6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:15.517191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-94d497dd-451d-4160-930e-af04bf947cc6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:15.575481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:15.629882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d497dd-451d-4160-930e-af04bf947cc6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:42:15.631357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d497dd-451d-4160-930e-af04bf947cc6 service glance] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:42:15.649867 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 579/1152] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:42:17.844024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f0f9d-508c-44ef-ab8f-0cca72f4276c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:17.844661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f0f9d-508c-44ef-ab8f-0cca72f4276c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:17.844661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f0f9d-508c-44ef-ab8f-0cca72f4276c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:17.845893 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 574/1153] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:42:17.851601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb842d-e240-41a7-af44-4e5c774fa9cb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:17.852063 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb842d-e240-41a7-af44-4e5c774fa9cb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:17.852164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb842d-e240-41a7-af44-4e5c774fa9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:17.920263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-16cb842d-e240-41a7-af44-4e5c774fa9cb None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:42:17.921445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-16cb842d-e240-41a7-af44-4e5c774fa9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:18.026276 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 580/1154] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:42:18.290369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f0f9d-508c-44ef-ab8f-0cca72f4276c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:42:18.292901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-596f0f9d-508c-44ef-ab8f-0cca72f4276c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:18.431485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f0f9d-508c-44ef-ab8f-0cca72f4276c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:42:18.481687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:18.528968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0e96a-8c92-4b3b-898e-5aca8cf540f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:42:18.529562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0e96a-8c92-4b3b-898e-5aca8cf540f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:18.529839 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0e96a-8c92-4b3b-898e-5aca8cf540f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:18.530110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0e96a-8c92-4b3b-898e-5aca8cf540f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:18.530759 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0e96a-8c92-4b3b-898e-5aca8cf540f8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:42:18.531800 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING py.warnings [None req-60a0e96a-8c92-4b3b-898e-5aca8cf540f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:42:18.531800 dv-cinder-master-3-smb devstack@keystone.service[87872]: warnings.warn(msg) Oct 24 08:42:18.531800 dv-cinder-master-3-smb devstack@keystone.service[87872]: Oct 24 08:42:18.532933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0e96a-8c92-4b3b-898e-5aca8cf540f8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:42:18.553187 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 575/1155] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Oct 24 08:42:18 2022] GET /identity/v3/projects => generated 2553 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:42:20.703521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc4d1e-3f44-476b-b885-70052a16474a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:20.704087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc4d1e-3f44-476b-b885-70052a16474a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:20.704087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc4d1e-3f44-476b-b885-70052a16474a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:20.705319 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 581/1156] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:42:20.710915 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d17b4-9b36-412a-b030-749cb08b5315 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:20.711378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d17b4-9b36-412a-b030-749cb08b5315 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:20.711481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d17b4-9b36-412a-b030-749cb08b5315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:20.785253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c68d17b4-9b36-412a-b030-749cb08b5315 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:42:20.786461 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c68d17b4-9b36-412a-b030-749cb08b5315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:20.887600 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 576/1157] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:42:21.181181 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc4d1e-3f44-476b-b885-70052a16474a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:42:21.182751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3adc4d1e-3f44-476b-b885-70052a16474a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:21.192799 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:21.333182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc4d1e-3f44-476b-b885-70052a16474a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:42:21.418364 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:21.451705 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f105c65-0205-48bd-af85-862add9c3e1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:42:21.452315 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f105c65-0205-48bd-af85-862add9c3e1a service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:21.452592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f105c65-0205-48bd-af85-862add9c3e1a service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:21.452864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f105c65-0205-48bd-af85-862add9c3e1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:21.454453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1f105c65-0205-48bd-af85-862add9c3e1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:21.545067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f105c65-0205-48bd-af85-862add9c3e1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:42:21.546672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f105c65-0205-48bd-af85-862add9c3e1a service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:42:21.566074 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 582/1158] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:42:23.874443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc3a7a-9ff0-4a74-858c-022b61c1beea None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:23.875423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc3a7a-9ff0-4a74-858c-022b61c1beea None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:23.875423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc3a7a-9ff0-4a74-858c-022b61c1beea None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:23.877546 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 577/1159] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:42:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:42:23.884064 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dacd7-38cc-4cfe-860b-8f3a8506b651 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:23.884861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dacd7-38cc-4cfe-860b-8f3a8506b651 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:23.885033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dacd7-38cc-4cfe-860b-8f3a8506b651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:23.964393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-dc8dacd7-38cc-4cfe-860b-8f3a8506b651 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:42:23.965631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dc8dacd7-38cc-4cfe-860b-8f3a8506b651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:24.014658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:24.073328 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 583/1160] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:42:24.361512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc3a7a-9ff0-4a74-858c-022b61c1beea None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:42:24.364116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-61bc3a7a-9ff0-4a74-858c-022b61c1beea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:24.391309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:24.483879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc3a7a-9ff0-4a74-858c-022b61c1beea None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:42:24.562204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809bfee-4cfa-4a95-a083-0119bf9e9450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:42:24.562753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809bfee-4cfa-4a95-a083-0119bf9e9450 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:24.563028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809bfee-4cfa-4a95-a083-0119bf9e9450 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:24.563299 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809bfee-4cfa-4a95-a083-0119bf9e9450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:24.564895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5809bfee-4cfa-4a95-a083-0119bf9e9450 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:24.644007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:24.689666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809bfee-4cfa-4a95-a083-0119bf9e9450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:42:24.691125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809bfee-4cfa-4a95-a083-0119bf9e9450 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:42:24.709557 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 578/1161] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:42:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:42:26.893339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55005df3-e110-40ee-975c-8e6bbfaa39b0 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:26.893946 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55005df3-e110-40ee-975c-8e6bbfaa39b0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:26.893946 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55005df3-e110-40ee-975c-8e6bbfaa39b0 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:26.895297 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 584/1162] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:42:26.901703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f72d2-e1be-4f7d-b8a7-7a0d6cb06f10 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:26.902454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f72d2-e1be-4f7d-b8a7-7a0d6cb06f10 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:26.902644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f72d2-e1be-4f7d-b8a7-7a0d6cb06f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:26.976752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a85f72d2-e1be-4f7d-b8a7-7a0d6cb06f10 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:42:26.977953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a85f72d2-e1be-4f7d-b8a7-7a0d6cb06f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:27.083399 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 579/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:42:27.368618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55005df3-e110-40ee-975c-8e6bbfaa39b0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:42:27.371098 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-55005df3-e110-40ee-975c-8e6bbfaa39b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:27.487753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55005df3-e110-40ee-975c-8e6bbfaa39b0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:42:27.559527 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:27.561154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960e974-4c30-4206-8656-ca3b2fba967f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:42:27.561801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960e974-4c30-4206-8656-ca3b2fba967f service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:27.562036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960e974-4c30-4206-8656-ca3b2fba967f service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:27.562553 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960e974-4c30-4206-8656-ca3b2fba967f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:27.564460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3960e974-4c30-4206-8656-ca3b2fba967f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:27.674525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960e974-4c30-4206-8656-ca3b2fba967f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:42:27.675978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960e974-4c30-4206-8656-ca3b2fba967f service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:42:27.694642 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 585/1164] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:42:29.923957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec41ec-bbbe-4797-8932-08949413668a None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:29.924539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec41ec-bbbe-4797-8932-08949413668a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:29.924539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec41ec-bbbe-4797-8932-08949413668a None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:29.925865 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 580/1165] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:42:29.933754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a1dff-c36a-4ccc-a3ee-2443ce3f1b21 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:29.934206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a1dff-c36a-4ccc-a3ee-2443ce3f1b21 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:29.934314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a1dff-c36a-4ccc-a3ee-2443ce3f1b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:30.004657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e02a1dff-c36a-4ccc-a3ee-2443ce3f1b21 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:42:30.005862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e02a1dff-c36a-4ccc-a3ee-2443ce3f1b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:30.061043 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:30.109054 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 586/1166] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:42:30.402467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec41ec-bbbe-4797-8932-08949413668a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:42:30.404361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fbec41ec-bbbe-4797-8932-08949413668a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:30.442555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:30.521253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec41ec-bbbe-4797-8932-08949413668a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:42:30.596407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05767daa-b538-4c88-a597-1dca3316c212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:42:30.597013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05767daa-b538-4c88-a597-1dca3316c212 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:30.597292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05767daa-b538-4c88-a597-1dca3316c212 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:30.597570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05767daa-b538-4c88-a597-1dca3316c212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:30.599218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-05767daa-b538-4c88-a597-1dca3316c212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:30.682386 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:30.712349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05767daa-b538-4c88-a597-1dca3316c212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:42:30.713776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05767daa-b538-4c88-a597-1dca3316c212 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:42:30.732197 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 581/1167] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:42:33.087418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca731ed2-b116-4734-b922-b64cbd08754e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:33.088327 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca731ed2-b116-4734-b922-b64cbd08754e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:33.088425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca731ed2-b116-4734-b922-b64cbd08754e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:33.090523 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 587/1168] 10.222.0.26 () {54 vars in 927 bytes} [Mon Oct 24 08:42:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:42:33.096156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee7ee4-1320-4190-883c-8d28afe7ea15 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:33.096579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee7ee4-1320-4190-883c-8d28afe7ea15 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:33.096706 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee7ee4-1320-4190-883c-8d28afe7ea15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:33.170607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-47ee7ee4-1320-4190-883c-8d28afe7ea15 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:42:33.171864 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-47ee7ee4-1320-4190-883c-8d28afe7ea15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:33.280214 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 582/1169] 10.222.0.26 () {58 vars in 1024 bytes} [Mon Oct 24 08:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:42:33.566987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca731ed2-b116-4734-b922-b64cbd08754e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:42:33.569479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ca731ed2-b116-4734-b922-b64cbd08754e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:33.681400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca731ed2-b116-4734-b922-b64cbd08754e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:42:33.758430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db891786-98f0-478d-a91d-6d521c4ba2bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:42:33.759329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891786-98f0-478d-a91d-6d521c4ba2bb service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:42:33.759604 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891786-98f0-478d-a91d-6d521c4ba2bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:42:33.759961 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db891786-98f0-478d-a91d-6d521c4ba2bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:42:33.762068 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-db891786-98f0-478d-a91d-6d521c4ba2bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:42:33.770839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:33.878677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db891786-98f0-478d-a91d-6d521c4ba2bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:42:33.880154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db891786-98f0-478d-a91d-6d521c4ba2bb service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:42:33.899373 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 588/1170] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:42:43.278603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=82 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:42:43.897186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=50 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:53.315408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c98cd-6263-426f-a854-4acf381a82a1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:53.315938 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c98cd-6263-426f-a854-4acf381a82a1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:53.315938 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c98cd-6263-426f-a854-4acf381a82a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:53.395730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-284c98cd-6263-426f-a854-4acf381a82a1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:43:53.396925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-284c98cd-6263-426f-a854-4acf381a82a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:53.495682 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 583/1171] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:43:53.503959 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cac1b7-3ba7-4d4b-826e-49612f2ba7ca None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:53.504493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cac1b7-3ba7-4d4b-826e-49612f2ba7ca None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:53.504493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cac1b7-3ba7-4d4b-826e-49612f2ba7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:53.583610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b1cac1b7-3ba7-4d4b-826e-49612f2ba7ca None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:43:53.585565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b1cac1b7-3ba7-4d4b-826e-49612f2ba7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:53.690933 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 589/1172] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:43:53.695716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c98cd-6263-426f-a854-4acf381a82a1 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:53.697025 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-284c98cd-6263-426f-a854-4acf381a82a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:53.708860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c98cd-6263-426f-a854-4acf381a82a1 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:53.772356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:53.807939 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:53.808542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:53.808817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:53.809086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:53.809733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:43:53.810038 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:53.811071 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING py.warnings [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:43:53.811071 dv-cinder-master-3-smb devstack@keystone.service[87872]: warnings.warn(msg) Oct 24 08:43:53.811071 dv-cinder-master-3-smb devstack@keystone.service[87872]: Oct 24 08:43:53.811947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:53.823321 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 584/1173] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:43:53.833061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cac1b7-3ba7-4d4b-826e-49612f2ba7ca None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:53.842690 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cac1b7-3ba7-4d4b-826e-49612f2ba7ca None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:53.906132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:53.941830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:53.942443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:53.942738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:53.943036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:53.943690 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:53.944801 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING py.warnings [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:43:53.944801 dv-cinder-master-3-smb devstack@keystone.service[87873]: warnings.warn(msg) Oct 24 08:43:53.944801 dv-cinder-master-3-smb devstack@keystone.service[87873]: Oct 24 08:43:53.945988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:54.002740 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 590/1174] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:43:53 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:43:54.007192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:54.008781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:54.046523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376060-5f55-4256-91ef-8542c453f966 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:54.141874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:54.142483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:54.142784 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:54.143056 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:54.143717 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:54.144716 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING py.warnings [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:43:54.144716 dv-cinder-master-3-smb devstack@keystone.service[87872]: warnings.warn(msg) Oct 24 08:43:54.144716 dv-cinder-master-3-smb devstack@keystone.service[87872]: Oct 24 08:43:54.145919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:54.201031 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 585/1175] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:43:54 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:43:54.205287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:54.206861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:54.271270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9ea92-81d7-44f8-aa59-43f2e9513535 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:54.365898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:54.378187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac22e0-5249-4696-9011-dae87214139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:54.378836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac22e0-5249-4696-9011-dae87214139b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:54.379115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac22e0-5249-4696-9011-dae87214139b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:54.379382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac22e0-5249-4696-9011-dae87214139b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:54.380024 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac22e0-5249-4696-9011-dae87214139b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:54.381629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac22e0-5249-4696-9011-dae87214139b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:54.394372 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 591/1176] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:43:54.398739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:54.400250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:54.411711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8bce1-3fae-4d89-a124-94998e5d2028 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:54.443024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:54.516898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d426a-5fb0-4a5b-bda1-524447e3f646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:54.517512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d426a-5fb0-4a5b-bda1-524447e3f646 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:54.517790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d426a-5fb0-4a5b-bda1-524447e3f646 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:54.518062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d426a-5fb0-4a5b-bda1-524447e3f646 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:54.518760 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d426a-5fb0-4a5b-bda1-524447e3f646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:54.520430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d426a-5fb0-4a5b-bda1-524447e3f646 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:54.533009 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 586/1177] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:43:54.537611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac22e0-5249-4696-9011-dae87214139b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:54.539153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6aac22e0-5249-4696-9011-dae87214139b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:54.574406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac22e0-5249-4696-9011-dae87214139b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:54.668265 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27261987-c3c2-4a9c-923a-e81e67c52837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:54.668917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27261987-c3c2-4a9c-923a-e81e67c52837 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:54.669187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27261987-c3c2-4a9c-923a-e81e67c52837 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:54.669461 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27261987-c3c2-4a9c-923a-e81e67c52837 admin admin] PATH_INFO: `/v3/projects/b019beafc2734565be2140419ffe86e3/users/4f692bcc94cf46fabdc715f1ded866fc/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:54.693039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27261987-c3c2-4a9c-923a-e81e67c52837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b019beafc2734565be2140419ffe86e3, user_id=4f692bcc94cf46fabdc715f1ded866fc, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:54.696535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27261987-c3c2-4a9c-923a-e81e67c52837 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:54.724723 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 592/1178] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:43:54 2022] PUT /identity/v3/projects/b019beafc2734565be2140419ffe86e3/users/4f692bcc94cf46fabdc715f1ded866fc/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:43:54.736129 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fa09f-04ea-4cd8-add9-d1bfda2c0f54 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:54.736420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fa09f-04ea-4cd8-add9-d1bfda2c0f54 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:54.736692 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fa09f-04ea-4cd8-add9-d1bfda2c0f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:54.795146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ec8fa09f-04ea-4cd8-add9-d1bfda2c0f54 None None] MFA Rules not processed for user `4f692bcc94cf46fabdc715f1ded866fc`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:43:54.796355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ec8fa09f-04ea-4cd8-add9-d1bfda2c0f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:54.880651 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 587/1179] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:43:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:43:54.899356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c57118-8f48-4e8c-a5b4-e1ffd5565114 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:54.899643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c57118-8f48-4e8c-a5b4-e1ffd5565114 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:54.899930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c57118-8f48-4e8c-a5b4-e1ffd5565114 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:54.901334 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 593/1180] 10.222.0.26 () {54 vars in 963 bytes} [Mon Oct 24 08:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:43:54.906378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d296d0e-2852-417e-8f67-0fe311b2a9c5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:54.906657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d296d0e-2852-417e-8f67-0fe311b2a9c5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:54.906960 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d296d0e-2852-417e-8f67-0fe311b2a9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:54.938915 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:54.983618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2d296d0e-2852-417e-8f67-0fe311b2a9c5 None None] MFA Rules not processed for user `6046d6440af84aa28aab47a018638e18`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:43:54.984790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2d296d0e-2852-417e-8f67-0fe311b2a9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:55.068206 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 588/1181] 10.222.0.26 () {58 vars in 1060 bytes} [Mon Oct 24 08:43:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:43:55.076420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c57118-8f48-4e8c-a5b4-e1ffd5565114 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:55.077941 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-38c57118-8f48-4e8c-a5b4-e1ffd5565114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:55.136746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:55.193677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c57118-8f48-4e8c-a5b4-e1ffd5565114 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:55.266520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:55.267158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:55.267430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:55.267699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:55.269303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:55.373190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:55.374674 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:55.378271 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:55.394195 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 594/1182] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:43:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:43:55.442884 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa884c92-0909-4353-88f2-6e21687f111f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:55.443359 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa884c92-0909-4353-88f2-6e21687f111f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:55.443465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa884c92-0909-4353-88f2-6e21687f111f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:55.444665 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 589/1183] 10.222.0.26 () {52 vars in 830 bytes} [Mon Oct 24 08:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:43:55.448447 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:55.449679 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:55.550544 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae483b8-3fb5-4873-86c3-dc06dd726d93 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:55.647958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:55.648522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:55.648839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:55.649112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] PATH_INFO: `/v3/users/4f692bcc94cf46fabdc715f1ded866fc` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:55.662345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f692bcc94cf46fabdc715f1ded866fc)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:55.663410 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING py.warnings [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:43:55.663410 dv-cinder-master-3-smb devstack@keystone.service[87873]: warnings.warn(msg) Oct 24 08:43:55.663410 dv-cinder-master-3-smb devstack@keystone.service[87873]: Oct 24 08:43:55.664626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:55.674925 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:55.736250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f692bcc94cf46fabdc715f1ded866fc'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:55.750344 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f692bcc94cf46fabdc715f1ded866fc'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:55.750934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f692bcc94cf46fabdc715f1ded866fc'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:55.762027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f692bcc94cf46fabdc715f1ded866fc'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:55.785784 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f692bcc94cf46fabdc715f1ded866fc'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:55.792115 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 595/1184] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:43:55 2022] DELETE /identity/v3/users/4f692bcc94cf46fabdc715f1ded866fc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:43:55.796894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b44194-0e07-4ac6-83a6-27f0d214fbfe None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:55.797178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b44194-0e07-4ac6-83a6-27f0d214fbfe None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:55.797454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b44194-0e07-4ac6-83a6-27f0d214fbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:55.857374 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-31b44194-0e07-4ac6-83a6-27f0d214fbfe None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:43:55.858582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-31b44194-0e07-4ac6-83a6-27f0d214fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:55.935985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:55.951840 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 590/1185] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:43:55.963568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:55.964795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:56.065921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407edda-597d-4575-8bd0-249b8687759f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:56.121515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:43:56.142144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64bb4-8bce-4395-995b-cd74fef66788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:56.142787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64bb4-8bce-4395-995b-cd74fef66788 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:56.143067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64bb4-8bce-4395-995b-cd74fef66788 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:56.143339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64bb4-8bce-4395-995b-cd74fef66788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:56.144930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-60e64bb4-8bce-4395-995b-cd74fef66788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:56.254989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e64bb4-8bce-4395-995b-cd74fef66788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:56.256361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e64bb4-8bce-4395-995b-cd74fef66788 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:56.274894 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 596/1186] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:43:56.717580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b44194-0e07-4ac6-83a6-27f0d214fbfe None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:43:56.718880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-31b44194-0e07-4ac6-83a6-27f0d214fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:43:56.730534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b44194-0e07-4ac6-83a6-27f0d214fbfe None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:43:56.834701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:43:56.835276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:43:56.835559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:43:56.835814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] PATH_INFO: `/v3/projects/b019beafc2734565be2140419ffe86e3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:43:56.847907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b019beafc2734565be2140419ffe86e3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:43:56.849059 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING py.warnings [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:43:56.849059 dv-cinder-master-3-smb devstack@keystone.service[87872]: warnings.warn(msg) Oct 24 08:43:56.849059 dv-cinder-master-3-smb devstack@keystone.service[87872]: Oct 24 08:43:56.850265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:43:56.912707 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b019beafc2734565be2140419ffe86e3', 'request_id': 'req-a328bfcc-6e48-47df-899e-bc9e7abceff9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:56.920159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b019beafc2734565be2140419ffe86e3', 'request_id': 'req-a328bfcc-6e48-47df-899e-bc9e7abceff9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:56.934403 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b019beafc2734565be2140419ffe86e3', 'request_id': 'req-a328bfcc-6e48-47df-899e-bc9e7abceff9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:56.961752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] The token cache is being invalidate because project b019beafc2734565be2140419ffe86e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:43:56.962036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:56.962544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a328bfcc-6e48-47df-899e-bc9e7abceff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:43:56.963926 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 591/1187] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:43:56 2022] DELETE /identity/v3/projects/b019beafc2734565be2140419ffe86e3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:44:06.272465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=62 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:44:06.959701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:45:29.383568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903bdc-b030-4e6f-9c86-5a72b6ceb930 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:29.384147 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903bdc-b030-4e6f-9c86-5a72b6ceb930 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:29.384147 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903bdc-b030-4e6f-9c86-5a72b6ceb930 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:29.385515 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 597/1188] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:45:29.390687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aac26b-277c-419d-981f-7b16076aefe3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:29.391174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aac26b-277c-419d-981f-7b16076aefe3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:29.391274 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aac26b-277c-419d-981f-7b16076aefe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:29.479470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-87aac26b-277c-419d-981f-7b16076aefe3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:29.480691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-87aac26b-277c-419d-981f-7b16076aefe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:29.600229 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 592/1189] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:45:29.758389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8903bdc-b030-4e6f-9c86-5a72b6ceb930 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:45:29.761825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d8903bdc-b030-4e6f-9c86-5a72b6ceb930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:29.883670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8903bdc-b030-4e6f-9c86-5a72b6ceb930 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:45:29.889703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72475780-2906-4d33-8c0b-45a76def1c20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:45:29.890307 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72475780-2906-4d33-8c0b-45a76def1c20 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:29.890665 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72475780-2906-4d33-8c0b-45a76def1c20 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:29.890942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72475780-2906-4d33-8c0b-45a76def1c20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:29.899173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72475780-2906-4d33-8c0b-45a76def1c20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:45:29.900762 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72475780-2906-4d33-8c0b-45a76def1c20 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:45:29.910171 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 598/1190] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:45:30.972040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c649975-e6eb-488c-aa46-12535d28ad40 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:30.972672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c649975-e6eb-488c-aa46-12535d28ad40 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:30.972672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c649975-e6eb-488c-aa46-12535d28ad40 None None] PATH_INFO: `/v3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:30.973965 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 593/1191] 10.222.0.36 () {54 vars in 925 bytes} [Mon Oct 24 08:45:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:45:30.981176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7098d81-06bf-4061-a220-280db258a951 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:30.981656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7098d81-06bf-4061-a220-280db258a951 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:30.981761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7098d81-06bf-4061-a220-280db258a951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:30.990662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d07bb-42a8-44b7-b9b0-11c38f2c5b3e None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:30.990989 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d07bb-42a8-44b7-b9b0-11c38f2c5b3e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:30.991224 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d07bb-42a8-44b7-b9b0-11c38f2c5b3e None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:30.992692 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 594/1192] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:45:30.997203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ddd83-65d3-4730-abba-209bc008cbbc None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:30.997486 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ddd83-65d3-4730-abba-209bc008cbbc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:30.997758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ddd83-65d3-4730-abba-209bc008cbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:31.018462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2a0ddd83-65d3-4730-abba-209bc008cbbc None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:31.019719 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2a0ddd83-65d3-4730-abba-209bc008cbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:31.032725 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 595/1193] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:45:31.051775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e7098d81-06bf-4061-a220-280db258a951 None None] MFA Rules not processed for user `c3749568726644be9ebdf4d87463ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:31.053079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e7098d81-06bf-4061-a220-280db258a951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:31.121504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:45:31.127814 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 599/1194] 10.222.0.36 () {58 vars in 1015 bytes} [Mon Oct 24 08:45:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:45:31.219045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ddd83-65d3-4730-abba-209bc008cbbc None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:45:31.236260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ddd83-65d3-4730-abba-209bc008cbbc None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:45:31.243230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c362a2-521c-4105-96e4-499c973ebd67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:45:31.243843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c362a2-521c-4105-96e4-499c973ebd67 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:31.244123 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c362a2-521c-4105-96e4-499c973ebd67 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:31.244396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c362a2-521c-4105-96e4-499c973ebd67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:31.253488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c362a2-521c-4105-96e4-499c973ebd67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:45:31.255108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c362a2-521c-4105-96e4-499c973ebd67 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:45:31.265357 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 596/1195] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:45:39.851351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de270c6-69e7-4621-a1ae-902f79064f9e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:39.851863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de270c6-69e7-4621-a1ae-902f79064f9e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:39.851863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de270c6-69e7-4621-a1ae-902f79064f9e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:39.853246 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 600/1196] 10.222.0.26 () {54 vars in 961 bytes} [Mon Oct 24 08:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:45:39.858414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fb4ca-0f61-440f-9c7c-de4546c5dd7a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:39.858928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fb4ca-0f61-440f-9c7c-de4546c5dd7a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:39.859043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fb4ca-0f61-440f-9c7c-de4546c5dd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:39.879729 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-be3fb4ca-0f61-440f-9c7c-de4546c5dd7a None None] MFA Rules not processed for user `c3749568726644be9ebdf4d87463ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:39.880984 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-be3fb4ca-0f61-440f-9c7c-de4546c5dd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:39.891505 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 597/1197] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Oct 24 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:45:39.899420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de270c6-69e7-4621-a1ae-902f79064f9e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:45:39.914865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de270c6-69e7-4621-a1ae-902f79064f9e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:45:39.920324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51082c3a-4ddd-4e35-828b-12e16a1dbf42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:45:39.920920 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51082c3a-4ddd-4e35-828b-12e16a1dbf42 service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:39.921218 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51082c3a-4ddd-4e35-828b-12e16a1dbf42 service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:39.921470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51082c3a-4ddd-4e35-828b-12e16a1dbf42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:39.930104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51082c3a-4ddd-4e35-828b-12e16a1dbf42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:45:39.931740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51082c3a-4ddd-4e35-828b-12e16a1dbf42 service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:45:39.941413 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 601/1198] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:45:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:45:42.730913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dee08-8d6a-4319-b7fa-46ddf0f74915 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:42.731823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dee08-8d6a-4319-b7fa-46ddf0f74915 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:42.731823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dee08-8d6a-4319-b7fa-46ddf0f74915 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:42.733975 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 598/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:45:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:45:42.739002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62564b-0697-437f-8136-8607b7b5e5fe None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:42.739463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62564b-0697-437f-8136-8607b7b5e5fe None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:42.739568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62564b-0697-437f-8136-8607b7b5e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:42.766290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4f62564b-0697-437f-8136-8607b7b5e5fe None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:42.767523 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4f62564b-0697-437f-8136-8607b7b5e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:42.776572 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 602/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:45:42.933503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5dee08-8d6a-4319-b7fa-46ddf0f74915 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:45:42.956376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5dee08-8d6a-4319-b7fa-46ddf0f74915 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:45:42.964089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30d1d1-f16f-423e-b3cf-abca8aeb95b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:45:42.964690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30d1d1-f16f-423e-b3cf-abca8aeb95b2 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:42.964968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30d1d1-f16f-423e-b3cf-abca8aeb95b2 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:42.965272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30d1d1-f16f-423e-b3cf-abca8aeb95b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:42.973993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30d1d1-f16f-423e-b3cf-abca8aeb95b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:45:42.976321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30d1d1-f16f-423e-b3cf-abca8aeb95b2 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:45:42.986417 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 599/1201] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:45:44.068273 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc5d0-3981-4bdb-b419-e78c2d4515bb None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:44.069190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc5d0-3981-4bdb-b419-e78c2d4515bb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:44.069190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc5d0-3981-4bdb-b419-e78c2d4515bb None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:44.071310 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 603/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 08:45:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:45:44.076518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f793e-7d9d-4702-b6aa-005a21c81461 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:44.076981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f793e-7d9d-4702-b6aa-005a21c81461 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:44.077090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f793e-7d9d-4702-b6aa-005a21c81461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:44.101525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-545f793e-7d9d-4702-b6aa-005a21c81461 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:44.103231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-545f793e-7d9d-4702-b6aa-005a21c81461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:44.112786 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 600/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 08:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:45:44.263899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bc5d0-3981-4bdb-b419-e78c2d4515bb None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:45:44.287518 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bc5d0-3981-4bdb-b419-e78c2d4515bb None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:45:44.295388 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68849-0c22-442e-a481-af5b52f2d6a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:45:44.295991 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68849-0c22-442e-a481-af5b52f2d6a4 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:44.296262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68849-0c22-442e-a481-af5b52f2d6a4 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:44.296525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68849-0c22-442e-a481-af5b52f2d6a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:44.304999 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68849-0c22-442e-a481-af5b52f2d6a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:45:44.306586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68849-0c22-442e-a481-af5b52f2d6a4 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:45:44.315897 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 604/1204] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:45:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:45:48.451001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ff1ef-b552-4f02-9e10-8c398e8deabb None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:48.451578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ff1ef-b552-4f02-9e10-8c398e8deabb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:48.451578 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ff1ef-b552-4f02-9e10-8c398e8deabb None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:48.452882 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 601/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:45:48.459207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fde5a0-ceae-43fe-9999-e78ee38f5a52 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:48.459971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fde5a0-ceae-43fe-9999-e78ee38f5a52 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:48.460108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fde5a0-ceae-43fe-9999-e78ee38f5a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:48.481780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c8fde5a0-ceae-43fe-9999-e78ee38f5a52 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:48.483025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c8fde5a0-ceae-43fe-9999-e78ee38f5a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:48.492247 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 605/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:45:48.784228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ff1ef-b552-4f02-9e10-8c398e8deabb None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:45:48.787376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1e3ff1ef-b552-4f02-9e10-8c398e8deabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:48.817983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:45:48.878888 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ff1ef-b552-4f02-9e10-8c398e8deabb None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:45:48.882575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f10b6-52c5-4a5a-8a84-896ce484df10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:45:48.883213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f10b6-52c5-4a5a-8a84-896ce484df10 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:48.883489 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f10b6-52c5-4a5a-8a84-896ce484df10 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:48.883760 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f10b6-52c5-4a5a-8a84-896ce484df10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:48.893017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f10b6-52c5-4a5a-8a84-896ce484df10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:45:48.894433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f10b6-52c5-4a5a-8a84-896ce484df10 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:45:48.903901 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 602/1207] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:45:51.033080 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd50a69-2129-4149-ad94-8e968103925c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:51.033664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd50a69-2129-4149-ad94-8e968103925c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:51.033664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd50a69-2129-4149-ad94-8e968103925c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:51.035031 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 606/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:45:51.041343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c07b9b-f1c3-45fe-a3a2-d8b0750c9389 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:51.041807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c07b9b-f1c3-45fe-a3a2-d8b0750c9389 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:51.041927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c07b9b-f1c3-45fe-a3a2-d8b0750c9389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:51.064747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-59c07b9b-f1c3-45fe-a3a2-d8b0750c9389 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:51.065958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-59c07b9b-f1c3-45fe-a3a2-d8b0750c9389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:51.075781 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 603/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:45:51.370527 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd50a69-2129-4149-ad94-8e968103925c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:45:51.382183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd50a69-2129-4149-ad94-8e968103925c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:45:51.386176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dfdb6-55c1-4e34-a05d-72786e0ee9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:45:51.386803 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dfdb6-55c1-4e34-a05d-72786e0ee9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:51.387083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dfdb6-55c1-4e34-a05d-72786e0ee9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:51.387357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dfdb6-55c1-4e34-a05d-72786e0ee9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:51.396767 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dfdb6-55c1-4e34-a05d-72786e0ee9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:45:51.398198 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dfdb6-55c1-4e34-a05d-72786e0ee9a5 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:45:51.408121 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 607/1210] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:45:55.255219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb5df2-cc79-4206-956b-0ca985b104c4 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:55.255842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb5df2-cc79-4206-956b-0ca985b104c4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:55.255842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb5df2-cc79-4206-956b-0ca985b104c4 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:55.257216 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 604/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:45:55.262617 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ef8ab-4aa2-42cf-84b8-f239f07ed619 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:55.263150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ef8ab-4aa2-42cf-84b8-f239f07ed619 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:55.263150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ef8ab-4aa2-42cf-84b8-f239f07ed619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:55.282703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d27ef8ab-4aa2-42cf-84b8-f239f07ed619 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:55.283919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d27ef8ab-4aa2-42cf-84b8-f239f07ed619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:55.292792 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 608/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:45:55.593671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb5df2-cc79-4206-956b-0ca985b104c4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:45:55.595754 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-86bb5df2-cc79-4206-956b-0ca985b104c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:55.677767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb5df2-cc79-4206-956b-0ca985b104c4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:45:55.681774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afdd8b-2333-4819-a1e3-f1cb35bb875b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:45:55.682379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afdd8b-2333-4819-a1e3-f1cb35bb875b service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:55.682658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afdd8b-2333-4819-a1e3-f1cb35bb875b service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:55.682986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afdd8b-2333-4819-a1e3-f1cb35bb875b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:55.692080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afdd8b-2333-4819-a1e3-f1cb35bb875b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:45:55.693499 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afdd8b-2333-4819-a1e3-f1cb35bb875b service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:45:55.702781 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 605/1213] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:45:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:45:57.804392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faedff-f42c-4d85-ba3b-64e2ac7900b0 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:57.805272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faedff-f42c-4d85-ba3b-64e2ac7900b0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:57.805272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faedff-f42c-4d85-ba3b-64e2ac7900b0 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:57.807431 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 609/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 08:45:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:45:57.813874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744e923-5034-481e-9c34-24ecbd4c8dcd None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:57.814649 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744e923-5034-481e-9c34-24ecbd4c8dcd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:57.814854 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744e923-5034-481e-9c34-24ecbd4c8dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:57.838447 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6744e923-5034-481e-9c34-24ecbd4c8dcd None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:45:57.839733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6744e923-5034-481e-9c34-24ecbd4c8dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:57.849078 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 606/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 08:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:45:58.145805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0faedff-f42c-4d85-ba3b-64e2ac7900b0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:45:58.149123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b0faedff-f42c-4d85-ba3b-64e2ac7900b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:45:58.165208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0faedff-f42c-4d85-ba3b-64e2ac7900b0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:45:58.169545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e2f73-d542-4a99-a162-2082b9195d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:45:58.170140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e2f73-d542-4a99-a162-2082b9195d31 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:45:58.170416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e2f73-d542-4a99-a162-2082b9195d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:45:58.170677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e2f73-d542-4a99-a162-2082b9195d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:45:58.181163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e2f73-d542-4a99-a162-2082b9195d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:45:58.182590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e2f73-d542-4a99-a162-2082b9195d31 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:45:58.192380 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 610/1216] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:45:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:46:07.833361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=66 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:46:08.188194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=34 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:31.652780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60ba4-5fa8-4f31-8314-a3e256e71b35 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:31.664248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60ba4-5fa8-4f31-8314-a3e256e71b35 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:31.664248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60ba4-5fa8-4f31-8314-a3e256e71b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:31.679765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-92a60ba4-5fa8-4f31-8314-a3e256e71b35 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:31.680991 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-92a60ba4-5fa8-4f31-8314-a3e256e71b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:31.694574 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 607/1217] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:31.705765 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321582d-6576-4ecb-b6b7-3942c74198b6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:31.707020 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321582d-6576-4ecb-b6b7-3942c74198b6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:31.707543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321582d-6576-4ecb-b6b7-3942c74198b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:31.740151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8321582d-6576-4ecb-b6b7-3942c74198b6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:31.741775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8321582d-6576-4ecb-b6b7-3942c74198b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:31.758302 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 611/1218] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:31.763580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a60ba4-5fa8-4f31-8314-a3e256e71b35 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:31.781845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a60ba4-5fa8-4f31-8314-a3e256e71b35 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:31.791663 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002aa669-6037-4458-903f-8b80c9e27eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:31.792304 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002aa669-6037-4458-903f-8b80c9e27eac admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:31.792575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002aa669-6037-4458-903f-8b80c9e27eac admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:31.792843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002aa669-6037-4458-903f-8b80c9e27eac admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:31.793509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002aa669-6037-4458-903f-8b80c9e27eac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:31.793806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002aa669-6037-4458-903f-8b80c9e27eac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:31.795478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002aa669-6037-4458-903f-8b80c9e27eac admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:31.817191 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 608/1219] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:31.826329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321582d-6576-4ecb-b6b7-3942c74198b6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:31.840721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321582d-6576-4ecb-b6b7-3942c74198b6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:31.848591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2a138-0f5a-4bfd-9bca-94abe3094e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:31.849597 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2a138-0f5a-4bfd-9bca-94abe3094e4a admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:31.850120 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2a138-0f5a-4bfd-9bca-94abe3094e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:31.850610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2a138-0f5a-4bfd-9bca-94abe3094e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:31.851636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2a138-0f5a-4bfd-9bca-94abe3094e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:31.854546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2a138-0f5a-4bfd-9bca-94abe3094e4a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:31.918134 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 612/1220] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:31 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:31.922899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002aa669-6037-4458-903f-8b80c9e27eac admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:31.940302 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002aa669-6037-4458-903f-8b80c9e27eac admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.028352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3fc40-a1e1-4d10-8821-399c030bc87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.028981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3fc40-a1e1-4d10-8821-399c030bc87b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.029254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3fc40-a1e1-4d10-8821-399c030bc87b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.029520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3fc40-a1e1-4d10-8821-399c030bc87b admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.030164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3fc40-a1e1-4d10-8821-399c030bc87b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.032435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3fc40-a1e1-4d10-8821-399c030bc87b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.056894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d4b8a-417f-4df9-856d-55e39b9c1c42 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.057581 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d4b8a-417f-4df9-856d-55e39b9c1c42 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.057706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d4b8a-417f-4df9-856d-55e39b9c1c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.063305 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 609/1221] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:31 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:32.073259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3fc40-a1e1-4d10-8821-399c030bc87b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.087540 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3fc40-a1e1-4d10-8821-399c030bc87b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.100716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64038d-02da-49ba-a109-62fa10cb2944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.101396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64038d-02da-49ba-a109-62fa10cb2944 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.101697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64038d-02da-49ba-a109-62fa10cb2944 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.101969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64038d-02da-49ba-a109-62fa10cb2944 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.102585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-aa4d4b8a-417f-4df9-856d-55e39b9c1c42 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.102878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64038d-02da-49ba-a109-62fa10cb2944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.104131 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aa4d4b8a-417f-4df9-856d-55e39b9c1c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.104571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64038d-02da-49ba-a109-62fa10cb2944 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.122771 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 613/1222] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.126607 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 610/1223] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:47:32.132183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d4b8a-417f-4df9-856d-55e39b9c1c42 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.136999 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96884507-af3d-4801-bd83-b1e13ca1d921 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.137271 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96884507-af3d-4801-bd83-b1e13ca1d921 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.137541 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96884507-af3d-4801-bd83-b1e13ca1d921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.149170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d4b8a-417f-4df9-856d-55e39b9c1c42 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.159177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491087b-e9ca-49af-a8cd-a84fe491ae74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.162049 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491087b-e9ca-49af-a8cd-a84fe491ae74 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.163017 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491087b-e9ca-49af-a8cd-a84fe491ae74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.163344 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491087b-e9ca-49af-a8cd-a84fe491ae74 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.164132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491087b-e9ca-49af-a8cd-a84fe491ae74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.166886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491087b-e9ca-49af-a8cd-a84fe491ae74 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.179821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-96884507-af3d-4801-bd83-b1e13ca1d921 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.180982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-96884507-af3d-4801-bd83-b1e13ca1d921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.186784 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 614/1224] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:32.188243 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe8c4d-20af-4d56-b1fa-205c08725915 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.188571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe8c4d-20af-4d56-b1fa-205c08725915 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.188847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe8c4d-20af-4d56-b1fa-205c08725915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.200033 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 611/1225] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:47:32.200960 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96884507-af3d-4801-bd83-b1e13ca1d921 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.220185 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96884507-af3d-4801-bd83-b1e13ca1d921 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.231609 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-27fe8c4d-20af-4d56-b1fa-205c08725915 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.233416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-27fe8c4d-20af-4d56-b1fa-205c08725915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.236221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950b02a-4030-4671-9c38-dd0e77647910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.236941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950b02a-4030-4671-9c38-dd0e77647910 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.237225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950b02a-4030-4671-9c38-dd0e77647910 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.237492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950b02a-4030-4671-9c38-dd0e77647910 admin admin] PATH_INFO: `/v3/projects/e3b77fd80fd84c29b4581fda1ff8cef9/users/7d793a1f26a3467ea8119e27e3394371/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.261252 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 615/1226] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.264895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe8c4d-20af-4d56-b1fa-205c08725915 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.266229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950b02a-4030-4671-9c38-dd0e77647910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b77fd80fd84c29b4581fda1ff8cef9, user_id=7d793a1f26a3467ea8119e27e3394371, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.279643 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1950b02a-4030-4671-9c38-dd0e77647910 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.295886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe8c4d-20af-4d56-b1fa-205c08725915 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.310333 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 612/1227] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:32 2022] PUT /identity/v3/projects/e3b77fd80fd84c29b4581fda1ff8cef9/users/7d793a1f26a3467ea8119e27e3394371/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:32.313052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7264bb-7a2e-4049-b776-092e316b23ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.313697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418d2d-ee3a-499c-b6e4-40bbbf4e6caf None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.314309 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7264bb-7a2e-4049-b776-092e316b23ef admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.314635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418d2d-ee3a-499c-b6e4-40bbbf4e6caf None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.314952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418d2d-ee3a-499c-b6e4-40bbbf4e6caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.316793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7264bb-7a2e-4049-b776-092e316b23ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.317112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7264bb-7a2e-4049-b776-092e316b23ef admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.318665 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7264bb-7a2e-4049-b776-092e316b23ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:32.319029 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7264bb-7a2e-4049-b776-092e316b23ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.320131 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING py.warnings [None req-ab7264bb-7a2e-4049-b776-092e316b23ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:47:32.320131 dv-cinder-master-3-smb devstack@keystone.service[87873]: warnings.warn(msg) Oct 24 08:47:32.320131 dv-cinder-master-3-smb devstack@keystone.service[87873]: Oct 24 08:47:32.323085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7264bb-7a2e-4049-b776-092e316b23ef admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.340348 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 616/1228] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:32.342274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed703ab-7001-48db-8ad0-f34ef3d70974 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.342802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed703ab-7001-48db-8ad0-f34ef3d70974 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.343315 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed703ab-7001-48db-8ad0-f34ef3d70974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.349255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-25418d2d-ee3a-499c-b6e4-40bbbf4e6caf None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.350446 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-25418d2d-ee3a-499c-b6e4-40bbbf4e6caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.365176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2ed703ab-7001-48db-8ad0-f34ef3d70974 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.368831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2ed703ab-7001-48db-8ad0-f34ef3d70974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.421013 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 613/1229] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.422875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59576e4-3743-430a-a54a-c2407915bf55 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.423182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59576e4-3743-430a-a54a-c2407915bf55 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.423450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59576e4-3743-430a-a54a-c2407915bf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.432844 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 617/1230] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.434100 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85707e2-4f9d-41f3-981a-a9eaae35831c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.434386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85707e2-4f9d-41f3-981a-a9eaae35831c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.434663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85707e2-4f9d-41f3-981a-a9eaae35831c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.445057 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b59576e4-3743-430a-a54a-c2407915bf55 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.446364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b59576e4-3743-430a-a54a-c2407915bf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.454644 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c85707e2-4f9d-41f3-981a-a9eaae35831c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.454891 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 614/1231] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.455847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c85707e2-4f9d-41f3-981a-a9eaae35831c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.456182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4718bc8-f26a-4575-9727-5677956fc3b3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.456469 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4718bc8-f26a-4575-9727-5677956fc3b3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.456804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4718bc8-f26a-4575-9727-5677956fc3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.463589 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 618/1232] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.464975 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120fe3c-89d9-4945-abd9-8be10516cba1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.465259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120fe3c-89d9-4945-abd9-8be10516cba1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.466425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120fe3c-89d9-4945-abd9-8be10516cba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.477394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c4718bc8-f26a-4575-9727-5677956fc3b3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.478646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c4718bc8-f26a-4575-9727-5677956fc3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.486788 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3120fe3c-89d9-4945-abd9-8be10516cba1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.487176 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 615/1233] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.488002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4718bc8-f26a-4575-9727-5677956fc3b3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.488130 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3120fe3c-89d9-4945-abd9-8be10516cba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.495926 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 619/1234] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.497144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a291d-bca8-49fb-bee7-7e02a2ccba85 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.497424 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a291d-bca8-49fb-bee7-7e02a2ccba85 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.497699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a291d-bca8-49fb-bee7-7e02a2ccba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.501086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4718bc8-f26a-4575-9727-5677956fc3b3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.506174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e249d61-31b6-4c5d-b5dc-3b7d05032738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.506969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e249d61-31b6-4c5d-b5dc-3b7d05032738 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.507343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e249d61-31b6-4c5d-b5dc-3b7d05032738 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.507620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e249d61-31b6-4c5d-b5dc-3b7d05032738 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.508278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e249d61-31b6-4c5d-b5dc-3b7d05032738 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.509357 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING py.warnings [None req-8e249d61-31b6-4c5d-b5dc-3b7d05032738 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:47:32.509357 dv-cinder-master-3-smb devstack@keystone.service[87872]: warnings.warn(msg) Oct 24 08:47:32.509357 dv-cinder-master-3-smb devstack@keystone.service[87872]: Oct 24 08:47:32.510601 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e249d61-31b6-4c5d-b5dc-3b7d05032738 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.516968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-253a291d-bca8-49fb-bee7-7e02a2ccba85 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.518163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-253a291d-bca8-49fb-bee7-7e02a2ccba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.525830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:32.526656 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 620/1235] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.530119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cd88c-7a5f-4370-a272-b788beeaaf5c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.530411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cd88c-7a5f-4370-a272-b788beeaaf5c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.530735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993cd88c-7a5f-4370-a272-b788beeaaf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.543874 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 616/1236] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:32.545129 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120411a5-954f-4bd9-87f1-17b1d9f552ee None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.545415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120411a5-954f-4bd9-87f1-17b1d9f552ee None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.545692 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120411a5-954f-4bd9-87f1-17b1d9f552ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.553952 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-993cd88c-7a5f-4370-a272-b788beeaaf5c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.555186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-993cd88c-7a5f-4370-a272-b788beeaaf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.565316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-120411a5-954f-4bd9-87f1-17b1d9f552ee None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.566524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-120411a5-954f-4bd9-87f1-17b1d9f552ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.612520 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 621/1237] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.613372 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cd88c-7a5f-4370-a272-b788beeaaf5c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.626104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993cd88c-7a5f-4370-a272-b788beeaaf5c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.626821 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 617/1238] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.627642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120411a5-954f-4bd9-87f1-17b1d9f552ee None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.631307 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f67be6-429a-4eff-96a6-53c0ab16323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.631943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f67be6-429a-4eff-96a6-53c0ab16323f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.632249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f67be6-429a-4eff-96a6-53c0ab16323f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.632504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f67be6-429a-4eff-96a6-53c0ab16323f admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.633159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f67be6-429a-4eff-96a6-53c0ab16323f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:32.633493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f67be6-429a-4eff-96a6-53c0ab16323f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.635164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f67be6-429a-4eff-96a6-53c0ab16323f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.636719 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:32.655423 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 622/1239] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:32.655423 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaefc4-7707-4d8a-8642-9ec9d7d85836 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.655423 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaefc4-7707-4d8a-8642-9ec9d7d85836 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.655423 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaefc4-7707-4d8a-8642-9ec9d7d85836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.671948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b1eaefc4-7707-4d8a-8642-9ec9d7d85836 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.673496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b1eaefc4-7707-4d8a-8642-9ec9d7d85836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.682031 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 623/1240] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.683421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd88891-2116-4cee-9684-1b30d4d674c2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.683645 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd88891-2116-4cee-9684-1b30d4d674c2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.683917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd88891-2116-4cee-9684-1b30d4d674c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.684471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120411a5-954f-4bd9-87f1-17b1d9f552ee None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.688216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d8d2a-8a38-4f82-863d-2806b278a267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.688819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d8d2a-8a38-4f82-863d-2806b278a267 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.689095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d8d2a-8a38-4f82-863d-2806b278a267 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.689476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d8d2a-8a38-4f82-863d-2806b278a267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.700726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d8d2a-8a38-4f82-863d-2806b278a267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.702174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d8d2a-8a38-4f82-863d-2806b278a267 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.704493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3bd88891-2116-4cee-9684-1b30d4d674c2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.705672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3bd88891-2116-4cee-9684-1b30d4d674c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.711647 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 618/1241] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:32.712803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ebfa5-0f9a-4107-a1b7-a82c9c2f5389 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.713077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ebfa5-0f9a-4107-a1b7-a82c9c2f5389 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.713347 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ebfa5-0f9a-4107-a1b7-a82c9c2f5389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.713817 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 624/1242] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.714561 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd88891-2116-4cee-9684-1b30d4d674c2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.724689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd88891-2116-4cee-9684-1b30d4d674c2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.729612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588cbc8-8fa4-4547-9691-1e50a2a21043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.730205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588cbc8-8fa4-4547-9691-1e50a2a21043 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.730482 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588cbc8-8fa4-4547-9691-1e50a2a21043 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.730774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588cbc8-8fa4-4547-9691-1e50a2a21043 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.731402 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588cbc8-8fa4-4547-9691-1e50a2a21043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.732419 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING py.warnings [None req-6588cbc8-8fa4-4547-9691-1e50a2a21043 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:47:32.732419 dv-cinder-master-3-smb devstack@keystone.service[87873]: warnings.warn(msg) Oct 24 08:47:32.732419 dv-cinder-master-3-smb devstack@keystone.service[87873]: Oct 24 08:47:32.733259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-266ebfa5-0f9a-4107-a1b7-a82c9c2f5389 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:32.733621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588cbc8-8fa4-4547-9691-1e50a2a21043 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.734437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-266ebfa5-0f9a-4107-a1b7-a82c9c2f5389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:32.743108 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 619/1243] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:32.748584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ebfa5-0f9a-4107-a1b7-a82c9c2f5389 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.759736 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ebfa5-0f9a-4107-a1b7-a82c9c2f5389 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.764672 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 625/1244] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:32.764672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588cbc8-8fa4-4547-9691-1e50a2a21043 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.764919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3779f1-feec-45b4-8475-771f461b6571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.764919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3779f1-feec-45b4-8475-771f461b6571 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.765234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3779f1-feec-45b4-8475-771f461b6571 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.765395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3779f1-feec-45b4-8475-771f461b6571 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.766031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3779f1-feec-45b4-8475-771f461b6571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:32.766327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3779f1-feec-45b4-8475-771f461b6571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.767976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3779f1-feec-45b4-8475-771f461b6571 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.774938 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588cbc8-8fa4-4547-9691-1e50a2a21043 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.785173 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 620/1245] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:32.785387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e31b6d-9ce3-4de3-9c3b-92d147a098a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.785387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e31b6d-9ce3-4de3-9c3b-92d147a098a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.785603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3779f1-feec-45b4-8475-771f461b6571 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.785703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e31b6d-9ce3-4de3-9c3b-92d147a098a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.785703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e31b6d-9ce3-4de3-9c3b-92d147a098a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.785703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e31b6d-9ce3-4de3-9c3b-92d147a098a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:32.785703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e31b6d-9ce3-4de3-9c3b-92d147a098a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.785703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e31b6d-9ce3-4de3-9c3b-92d147a098a9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.797914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3779f1-feec-45b4-8475-771f461b6571 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.806018 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 626/1246] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:32.806018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e31b6d-9ce3-4de3-9c3b-92d147a098a9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.811581 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78008f8e-333c-4525-b0ce-5e76365df75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.812286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78008f8e-333c-4525-b0ce-5e76365df75c admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.812635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78008f8e-333c-4525-b0ce-5e76365df75c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.812807 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e31b6d-9ce3-4de3-9c3b-92d147a098a9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.812976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78008f8e-333c-4525-b0ce-5e76365df75c admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.813808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78008f8e-333c-4525-b0ce-5e76365df75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.815907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78008f8e-333c-4525-b0ce-5e76365df75c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.817767 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe0f50-7a3a-494c-805b-c175ceadf548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.818379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe0f50-7a3a-494c-805b-c175ceadf548 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.818654 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe0f50-7a3a-494c-805b-c175ceadf548 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.818953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe0f50-7a3a-494c-805b-c175ceadf548 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.819727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe0f50-7a3a-494c-805b-c175ceadf548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:32.820070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe0f50-7a3a-494c-805b-c175ceadf548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.821769 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe0f50-7a3a-494c-805b-c175ceadf548 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.835511 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 627/1247] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:32.836612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe0f50-7a3a-494c-805b-c175ceadf548 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.849414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe0f50-7a3a-494c-805b-c175ceadf548 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.852863 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 621/1248] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/projects => generated 391 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:32.854231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78008f8e-333c-4525-b0ce-5e76365df75c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.857011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5ba92-4c0e-4ee3-a68b-78400d56498d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.857934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5ba92-4c0e-4ee3-a68b-78400d56498d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.858276 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5ba92-4c0e-4ee3-a68b-78400d56498d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.858795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5ba92-4c0e-4ee3-a68b-78400d56498d admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.859534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5ba92-4c0e-4ee3-a68b-78400d56498d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:32.859908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5ba92-4c0e-4ee3-a68b-78400d56498d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.861780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5ba92-4c0e-4ee3-a68b-78400d56498d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.880017 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 628/1249] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:32.881453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5ba92-4c0e-4ee3-a68b-78400d56498d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.893598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5ba92-4c0e-4ee3-a68b-78400d56498d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.935611 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:32.946769 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78008f8e-333c-4525-b0ce-5e76365df75c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.953579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8b498-fb77-43d0-b9c3-64fd53613590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.954864 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8b498-fb77-43d0-b9c3-64fd53613590 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.955215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8b498-fb77-43d0-b9c3-64fd53613590 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.955486 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8b498-fb77-43d0-b9c3-64fd53613590 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.956310 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8b498-fb77-43d0-b9c3-64fd53613590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:32.956994 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8b498-fb77-43d0-b9c3-64fd53613590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.959203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8b498-fb77-43d0-b9c3-64fd53613590 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.972413 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 622/1250] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:32.973119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60739acf-c1d7-428f-a8a5-30f4756c61f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.973821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60739acf-c1d7-428f-a8a5-30f4756c61f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.974161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60739acf-c1d7-428f-a8a5-30f4756c61f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.974499 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60739acf-c1d7-428f-a8a5-30f4756c61f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.975232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60739acf-c1d7-428f-a8a5-30f4756c61f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:32.976918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60739acf-c1d7-428f-a8a5-30f4756c61f7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:32.980544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8b498-fb77-43d0-b9c3-64fd53613590 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.989432 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 629/1251] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:32.990365 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60739acf-c1d7-428f-a8a5-30f4756c61f7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:32.991344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8b498-fb77-43d0-b9c3-64fd53613590 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:32.996272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ca920-86f4-4aae-93c0-b5a85c928e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:32.996884 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ca920-86f4-4aae-93c0-b5a85c928e03 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:32.997157 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ca920-86f4-4aae-93c0-b5a85c928e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:32.997422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ca920-86f4-4aae-93c0-b5a85c928e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:32.998051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ca920-86f4-4aae-93c0-b5a85c928e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.000116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ca920-86f4-4aae-93c0-b5a85c928e03 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.004080 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60739acf-c1d7-428f-a8a5-30f4756c61f7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.010923 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da096d31-00f2-4d62-ad3c-1a2efa58e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.011568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da096d31-00f2-4d62-ad3c-1a2efa58e147 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.011853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da096d31-00f2-4d62-ad3c-1a2efa58e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.012541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da096d31-00f2-4d62-ad3c-1a2efa58e147 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.013217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da096d31-00f2-4d62-ad3c-1a2efa58e147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.015268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da096d31-00f2-4d62-ad3c-1a2efa58e147 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.030268 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 623/1252] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.033974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ca920-86f4-4aae-93c0-b5a85c928e03 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.049372 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 630/1253] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:32 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.050238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da096d31-00f2-4d62-ad3c-1a2efa58e147 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.118424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ca920-86f4-4aae-93c0-b5a85c928e03 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.124591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7ddc2-461c-4e61-a57a-b3cc98df141a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.125145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:33.126400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7ddc2-461c-4e61-a57a-b3cc98df141a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.126678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7ddc2-461c-4e61-a57a-b3cc98df141a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.127160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7ddc2-461c-4e61-a57a-b3cc98df141a admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.127808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7ddc2-461c-4e61-a57a-b3cc98df141a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.129996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7ddc2-461c-4e61-a57a-b3cc98df141a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.131568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da096d31-00f2-4d62-ad3c-1a2efa58e147 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.138800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faf57f-7cbd-4c85-9bb2-9f5b9e8fb891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.139557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faf57f-7cbd-4c85-9bb2-9f5b9e8fb891 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.140115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faf57f-7cbd-4c85-9bb2-9f5b9e8fb891 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.140370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faf57f-7cbd-4c85-9bb2-9f5b9e8fb891 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.141385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2faf57f-7cbd-4c85-9bb2-9f5b9e8fb891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.144027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2faf57f-7cbd-4c85-9bb2-9f5b9e8fb891 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.170773 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 624/1254] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/projects => generated 389 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.171014 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 631/1255] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.171108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7ddc2-461c-4e61-a57a-b3cc98df141a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.171194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faf57f-7cbd-4c85-9bb2-9f5b9e8fb891 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.179706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faf57f-7cbd-4c85-9bb2-9f5b9e8fb891 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.273603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ca235-be7e-4775-86c6-3ce63b53863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.275376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ca235-be7e-4775-86c6-3ce63b53863e admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.275714 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ca235-be7e-4775-86c6-3ce63b53863e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.276052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ca235-be7e-4775-86c6-3ce63b53863e admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.276761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ca235-be7e-4775-86c6-3ce63b53863e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.279406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ca235-be7e-4775-86c6-3ce63b53863e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.290300 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7ddc2-461c-4e61-a57a-b3cc98df141a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.295127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0baec-28ea-4418-9f7f-f57e4b8e1808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.295742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0baec-28ea-4418-9f7f-f57e4b8e1808 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.296021 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0baec-28ea-4418-9f7f-f57e4b8e1808 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.296296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0baec-28ea-4418-9f7f-f57e4b8e1808 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.297199 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0baec-28ea-4418-9f7f-f57e4b8e1808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.298928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0baec-28ea-4418-9f7f-f57e4b8e1808 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.332284 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 632/1256] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/projects => generated 355 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.332284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ca235-be7e-4775-86c6-3ce63b53863e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.337010 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 625/1257] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.337010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0baec-28ea-4418-9f7f-f57e4b8e1808 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.381240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:33.399113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ca235-be7e-4775-86c6-3ce63b53863e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.404099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba21f19-eb6e-47ae-ae81-8194e00d3ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.404914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba21f19-eb6e-47ae-ae81-8194e00d3ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.405047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0baec-28ea-4418-9f7f-f57e4b8e1808 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.405478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba21f19-eb6e-47ae-ae81-8194e00d3ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.406019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba21f19-eb6e-47ae-ae81-8194e00d3ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.406916 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba21f19-eb6e-47ae-ae81-8194e00d3ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.408661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba21f19-eb6e-47ae-ae81-8194e00d3ab7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.410051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eacbdb-826a-422c-ac1c-b82dc706390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.410637 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eacbdb-826a-422c-ac1c-b82dc706390b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.410944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eacbdb-826a-422c-ac1c-b82dc706390b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.411240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eacbdb-826a-422c-ac1c-b82dc706390b admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.411871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eacbdb-826a-422c-ac1c-b82dc706390b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.413895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eacbdb-826a-422c-ac1c-b82dc706390b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.423926 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 633/1258] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.423926 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba21f19-eb6e-47ae-ae81-8194e00d3ab7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.438171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba21f19-eb6e-47ae-ae81-8194e00d3ab7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.445173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b36ca-2686-49aa-93ec-4500b71954c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.445173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b36ca-2686-49aa-93ec-4500b71954c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.445173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b36ca-2686-49aa-93ec-4500b71954c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.445173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b36ca-2686-49aa-93ec-4500b71954c8 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.445907 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b36ca-2686-49aa-93ec-4500b71954c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.447206 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 626/1259] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.447534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b36ca-2686-49aa-93ec-4500b71954c8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.448070 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eacbdb-826a-422c-ac1c-b82dc706390b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.459880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eacbdb-826a-422c-ac1c-b82dc706390b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.465584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33382ee9-fc78-4c73-a49c-f1a31f73f3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.466243 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33382ee9-fc78-4c73-a49c-f1a31f73f3fb admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.466522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33382ee9-fc78-4c73-a49c-f1a31f73f3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.467420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33382ee9-fc78-4c73-a49c-f1a31f73f3fb admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.467669 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33382ee9-fc78-4c73-a49c-f1a31f73f3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.469580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33382ee9-fc78-4c73-a49c-f1a31f73f3fb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.476385 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 634/1260] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/users => generated 283 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.476727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b36ca-2686-49aa-93ec-4500b71954c8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.487758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b36ca-2686-49aa-93ec-4500b71954c8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.494837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518a084-16ea-4247-a29f-86e16cec89ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.495595 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518a084-16ea-4247-a29f-86e16cec89ca admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.495880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518a084-16ea-4247-a29f-86e16cec89ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.496162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518a084-16ea-4247-a29f-86e16cec89ca admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.496949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518a084-16ea-4247-a29f-86e16cec89ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.500283 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 627/1261] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/users => generated 295 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.500283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33382ee9-fc78-4c73-a49c-f1a31f73f3fb admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.500484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518a084-16ea-4247-a29f-86e16cec89ca admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.514110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33382ee9-fc78-4c73-a49c-f1a31f73f3fb admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.518455 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 635/1262] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.520266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518a084-16ea-4247-a29f-86e16cec89ca admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.521378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f739f44-abca-4d13-9a0a-0341cfe7ad6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.522097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f739f44-abca-4d13-9a0a-0341cfe7ad6f admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.523170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f739f44-abca-4d13-9a0a-0341cfe7ad6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.523460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f739f44-abca-4d13-9a0a-0341cfe7ad6f admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.524150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f739f44-abca-4d13-9a0a-0341cfe7ad6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.526134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f739f44-abca-4d13-9a0a-0341cfe7ad6f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.533392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518a084-16ea-4247-a29f-86e16cec89ca admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.540761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f75f-fa2e-48a1-8320-3358bb50540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.541830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f75f-fa2e-48a1-8320-3358bb50540f admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.542483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f75f-fa2e-48a1-8320-3358bb50540f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.542690 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f75f-fa2e-48a1-8320-3358bb50540f admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.543615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76f75f-fa2e-48a1-8320-3358bb50540f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.545760 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76f75f-fa2e-48a1-8320-3358bb50540f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.559680 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 628/1263] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/users => generated 278 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.564411 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f739f44-abca-4d13-9a0a-0341cfe7ad6f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.577316 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 636/1264] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/users => generated 293 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:33.578339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f75f-fa2e-48a1-8320-3358bb50540f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.580759 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f739f44-abca-4d13-9a0a-0341cfe7ad6f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.587093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714652f-7d28-4006-b51b-f35e4a106c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.588070 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714652f-7d28-4006-b51b-f35e4a106c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.589061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714652f-7d28-4006-b51b-f35e4a106c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.589512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714652f-7d28-4006-b51b-f35e4a106c65 admin admin] PATH_INFO: `/v3/projects/35e1a122b4354e2880b363d6f3815278/users/c300409c40db4636817fd34bfa7c4c77/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.594651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f75f-fa2e-48a1-8320-3358bb50540f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.600491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d97e0-6f47-47d2-9a6c-d9655cbbe7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.601291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d97e0-6f47-47d2-9a6c-d9655cbbe7ff admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.601816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d97e0-6f47-47d2-9a6c-d9655cbbe7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.602469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d97e0-6f47-47d2-9a6c-d9655cbbe7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.603407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d97e0-6f47-47d2-9a6c-d9655cbbe7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.605102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d97e0-6f47-47d2-9a6c-d9655cbbe7ff admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.611805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714652f-7d28-4006-b51b-f35e4a106c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e1a122b4354e2880b363d6f3815278, user_id=c300409c40db4636817fd34bfa7c4c77, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.614542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714652f-7d28-4006-b51b-f35e4a106c65 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.616193 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 637/1265] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.620028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d97e0-6f47-47d2-9a6c-d9655cbbe7ff admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.631186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d97e0-6f47-47d2-9a6c-d9655cbbe7ff admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.638630 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 629/1266] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:33 2022] PUT /identity/v3/projects/35e1a122b4354e2880b363d6f3815278/users/c300409c40db4636817fd34bfa7c4c77/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:33.639625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714652f-7d28-4006-b51b-f35e4a106c65 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.640319 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:33.702059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f259a-cfdf-4c99-8502-c7c1105db3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.702794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f259a-cfdf-4c99-8502-c7c1105db3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.703177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f259a-cfdf-4c99-8502-c7c1105db3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.703565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f259a-cfdf-4c99-8502-c7c1105db3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.704312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f259a-cfdf-4c99-8502-c7c1105db3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.706939 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f259a-cfdf-4c99-8502-c7c1105db3a5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.721812 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 638/1267] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.723535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f259a-cfdf-4c99-8502-c7c1105db3a5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.737552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f259a-cfdf-4c99-8502-c7c1105db3a5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.742120 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714652f-7d28-4006-b51b-f35e4a106c65 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.748464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265ebdc-d146-4ccc-9e60-4309f7d6d238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.749381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265ebdc-d146-4ccc-9e60-4309f7d6d238 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.750308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265ebdc-d146-4ccc-9e60-4309f7d6d238 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.750586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265ebdc-d146-4ccc-9e60-4309f7d6d238 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.751625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265ebdc-d146-4ccc-9e60-4309f7d6d238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.755011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260854d-365e-4fef-a512-cc10eb4e70f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.755799 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260854d-365e-4fef-a512-cc10eb4e70f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.756226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260854d-365e-4fef-a512-cc10eb4e70f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.756503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260854d-365e-4fef-a512-cc10eb4e70f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.757205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260854d-365e-4fef-a512-cc10eb4e70f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.758661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265ebdc-d146-4ccc-9e60-4309f7d6d238 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.762194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260854d-365e-4fef-a512-cc10eb4e70f0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.775696 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 639/1269] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.775896 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 630/1269] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.775896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260854d-365e-4fef-a512-cc10eb4e70f0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.776083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265ebdc-d146-4ccc-9e60-4309f7d6d238 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.785289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265ebdc-d146-4ccc-9e60-4309f7d6d238 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.789085 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260854d-365e-4fef-a512-cc10eb4e70f0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.791351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555301bc-75ce-467c-8240-a2e0589d6c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.791839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555301bc-75ce-467c-8240-a2e0589d6c26 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.792118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555301bc-75ce-467c-8240-a2e0589d6c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.792393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555301bc-75ce-467c-8240-a2e0589d6c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.793035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555301bc-75ce-467c-8240-a2e0589d6c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.794768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555301bc-75ce-467c-8240-a2e0589d6c26 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.798108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a45a56-63d5-4649-b636-653da08e0cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.798733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a45a56-63d5-4649-b636-653da08e0cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.799025 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a45a56-63d5-4649-b636-653da08e0cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.799289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a45a56-63d5-4649-b636-653da08e0cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.799932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a45a56-63d5-4649-b636-653da08e0cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.801547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a45a56-63d5-4649-b636-653da08e0cf4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.812489 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 631/1270] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.813332 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a45a56-63d5-4649-b636-653da08e0cf4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.816724 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 640/1271] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.818041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26730dcc-2a6c-497a-8264-26ac64503e2a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.818330 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26730dcc-2a6c-497a-8264-26ac64503e2a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.818611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26730dcc-2a6c-497a-8264-26ac64503e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.826882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a45a56-63d5-4649-b636-653da08e0cf4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.832122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca165e-9553-43b5-9092-8303a766f1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.832920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca165e-9553-43b5-9092-8303a766f1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.833214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca165e-9553-43b5-9092-8303a766f1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.833599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca165e-9553-43b5-9092-8303a766f1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.834463 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca165e-9553-43b5-9092-8303a766f1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.836162 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca165e-9553-43b5-9092-8303a766f1b0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.852898 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 632/1272] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.853651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca165e-9553-43b5-9092-8303a766f1b0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.864803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca165e-9553-43b5-9092-8303a766f1b0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.868232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-26730dcc-2a6c-497a-8264-26ac64503e2a None None] MFA Rules not processed for user `c300409c40db4636817fd34bfa7c4c77`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:33.869690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baa11a-7919-4e28-8d4c-adc945ad6552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.870288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa11a-7919-4e28-8d4c-adc945ad6552 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.870567 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa11a-7919-4e28-8d4c-adc945ad6552 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.870883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa11a-7919-4e28-8d4c-adc945ad6552 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.871292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-26730dcc-2a6c-497a-8264-26ac64503e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:33.871997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baa11a-7919-4e28-8d4c-adc945ad6552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.874141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baa11a-7919-4e28-8d4c-adc945ad6552 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.891439 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 633/1273] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.892690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baa11a-7919-4e28-8d4c-adc945ad6552 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.903412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baa11a-7919-4e28-8d4c-adc945ad6552 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.908300 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bef0fc-b7e3-4f2a-800d-8a05afe00746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.908937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bef0fc-b7e3-4f2a-800d-8a05afe00746 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.909221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bef0fc-b7e3-4f2a-800d-8a05afe00746 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.909494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bef0fc-b7e3-4f2a-800d-8a05afe00746 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.910444 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bef0fc-b7e3-4f2a-800d-8a05afe00746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.912084 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bef0fc-b7e3-4f2a-800d-8a05afe00746 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.924101 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 634/1274] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:33.924993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bef0fc-b7e3-4f2a-800d-8a05afe00746 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.937274 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bef0fc-b7e3-4f2a-800d-8a05afe00746 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:33.946808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c592f-d775-4d0a-b5aa-2f189562dd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:33.947797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c592f-d775-4d0a-b5aa-2f189562dd66 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:33.955212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c592f-d775-4d0a-b5aa-2f189562dd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:33.955501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c592f-d775-4d0a-b5aa-2f189562dd66 admin admin] PATH_INFO: `/v3/projects/f063c719bcda47e6874437d32d284815/users/8695364e58b04d829b12ae1b49757536/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:33.961783 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 641/1275] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:33.962589 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26730dcc-2a6c-497a-8264-26ac64503e2a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:33.990564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c592f-d775-4d0a-b5aa-2f189562dd66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f063c719bcda47e6874437d32d284815, user_id=8695364e58b04d829b12ae1b49757536, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:33.996580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c592f-d775-4d0a-b5aa-2f189562dd66 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:33.999871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26730dcc-2a6c-497a-8264-26ac64503e2a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:34.001758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:34.010836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50987738-9fd2-462f-ae87-4924cd0aa2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:34.011818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50987738-9fd2-462f-ae87-4924cd0aa2dc admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.012334 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50987738-9fd2-462f-ae87-4924cd0aa2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.012836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50987738-9fd2-462f-ae87-4924cd0aa2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.013701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50987738-9fd2-462f-ae87-4924cd0aa2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:34.015891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50987738-9fd2-462f-ae87-4924cd0aa2dc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:34.022126 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 635/1276] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:33 2022] PUT /identity/v3/projects/f063c719bcda47e6874437d32d284815/users/8695364e58b04d829b12ae1b49757536/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:34.023401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c592f-d775-4d0a-b5aa-2f189562dd66 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:34.039053 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 642/1277] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:34.040374 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50987738-9fd2-462f-ae87-4924cd0aa2dc admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:34.052874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50987738-9fd2-462f-ae87-4924cd0aa2dc admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:34.110179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c592f-d775-4d0a-b5aa-2f189562dd66 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:34.115624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e9dd1-f436-4337-b553-d0a49a83b8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:34.116240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e9dd1-f436-4337-b553-d0a49a83b8cc admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.116521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e9dd1-f436-4337-b553-d0a49a83b8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.116796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e9dd1-f436-4337-b553-d0a49a83b8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.117679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e9dd1-f436-4337-b553-d0a49a83b8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:34.119451 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e9dd1-f436-4337-b553-d0a49a83b8cc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:34.127403 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6750b-9c85-41a3-99f9-e8e9f1ec588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:34.128255 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6750b-9c85-41a3-99f9-e8e9f1ec588d admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.128830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6750b-9c85-41a3-99f9-e8e9f1ec588d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.128830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6750b-9c85-41a3-99f9-e8e9f1ec588d admin admin] PATH_INFO: `/v3/projects/7fa4a0e9972c4eb1963334ba4ae99a4f/users/b002748b80db40358571017d6b42ed09/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.130642 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 636/1278] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:34.131830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e9dd1-f436-4337-b553-d0a49a83b8cc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:34.146907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e9dd1-f436-4337-b553-d0a49a83b8cc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:34.147425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:34.152947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695c043-dca5-444f-9b0b-8bad4063e934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:34.155828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695c043-dca5-444f-9b0b-8bad4063e934 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.155828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695c043-dca5-444f-9b0b-8bad4063e934 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.155828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695c043-dca5-444f-9b0b-8bad4063e934 admin admin] PATH_INFO: `/v3/projects/fd5ce90003ca44b69bb7f58333ca1f71/users/678ce12a88374b1599c5bb89075a68cf/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.156677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6750b-9c85-41a3-99f9-e8e9f1ec588d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa4a0e9972c4eb1963334ba4ae99a4f, user_id=b002748b80db40358571017d6b42ed09, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:34.165845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6750b-9c85-41a3-99f9-e8e9f1ec588d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:34.183630 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 643/1279] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:34 2022] PUT /identity/v3/projects/7fa4a0e9972c4eb1963334ba4ae99a4f/users/b002748b80db40358571017d6b42ed09/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:34.184429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6750b-9c85-41a3-99f9-e8e9f1ec588d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:34.194292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695c043-dca5-444f-9b0b-8bad4063e934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5ce90003ca44b69bb7f58333ca1f71, user_id=678ce12a88374b1599c5bb89075a68cf, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:34.199711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695c043-dca5-444f-9b0b-8bad4063e934 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:34.226604 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 637/1280] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:34 2022] PUT /identity/v3/projects/fd5ce90003ca44b69bb7f58333ca1f71/users/678ce12a88374b1599c5bb89075a68cf/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:34.228406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc185adc-7d88-47aa-ba4f-c5b4faa6818f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.228859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc185adc-7d88-47aa-ba4f-c5b4faa6818f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.230518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc185adc-7d88-47aa-ba4f-c5b4faa6818f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.238793 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 638/1281] 10.222.0.26 () {54 vars in 961 bytes} [Mon Oct 24 08:47:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:47:34.240606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607996f-caca-4c6c-bad1-9ab66cff0aed None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.240606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607996f-caca-4c6c-bad1-9ab66cff0aed None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.240790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607996f-caca-4c6c-bad1-9ab66cff0aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.261608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5607996f-caca-4c6c-bad1-9ab66cff0aed None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:34.262899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5607996f-caca-4c6c-bad1-9ab66cff0aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:34.274017 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6750b-9c85-41a3-99f9-e8e9f1ec588d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:34.342914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a5202-ea6b-485d-8f9b-aed81af62028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:34.343360 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 639/1282] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:34.344070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a5202-ea6b-485d-8f9b-aed81af62028 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.344223 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607996f-caca-4c6c-bad1-9ab66cff0aed None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:34.344851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a5202-ea6b-485d-8f9b-aed81af62028 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.345279 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a5202-ea6b-485d-8f9b-aed81af62028 admin admin] PATH_INFO: `/v3/projects/5bca750f880a44988f7324b2f8da3e7b/users/43af1b9b66804624aff6eef2f435d581/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.357535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607996f-caca-4c6c-bad1-9ab66cff0aed None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:34.364867 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae95128-916b-4be9-bbe9-1c5e2b912d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:34.365239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a5202-ea6b-485d-8f9b-aed81af62028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bca750f880a44988f7324b2f8da3e7b, user_id=43af1b9b66804624aff6eef2f435d581, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:34.365952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae95128-916b-4be9-bbe9-1c5e2b912d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.366381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae95128-916b-4be9-bbe9-1c5e2b912d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.366859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae95128-916b-4be9-bbe9-1c5e2b912d56 admin admin] PATH_INFO: `/v3/projects/02be1b30677f400788b35b03938e44e5/users/712e5cf4cc004918817ab1802b9c69d6/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.367985 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a5202-ea6b-485d-8f9b-aed81af62028 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:34.384460 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 644/1283] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:34 2022] PUT /identity/v3/projects/5bca750f880a44988f7324b2f8da3e7b/users/43af1b9b66804624aff6eef2f435d581/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:34.385458 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a5202-ea6b-485d-8f9b-aed81af62028 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:34.392173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae95128-916b-4be9-bbe9-1c5e2b912d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02be1b30677f400788b35b03938e44e5, user_id=712e5cf4cc004918817ab1802b9c69d6, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:34.398178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae95128-916b-4be9-bbe9-1c5e2b912d56 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:34.417645 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 640/1284] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:34 2022] PUT /identity/v3/projects/02be1b30677f400788b35b03938e44e5/users/712e5cf4cc004918817ab1802b9c69d6/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:34.419456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412973b-a939-4239-90d3-9368201f2609 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.419743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412973b-a939-4239-90d3-9368201f2609 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.420017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412973b-a939-4239-90d3-9368201f2609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.469416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a5202-ea6b-485d-8f9b-aed81af62028 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:34.478686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0412973b-a939-4239-90d3-9368201f2609 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:34.479893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0412973b-a939-4239-90d3-9368201f2609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:34.494973 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:34.541456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:34.552770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4720c5-50a1-48dc-9836-64b0f1ec1a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:34.553543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4720c5-50a1-48dc-9836-64b0f1ec1a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.553916 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4720c5-50a1-48dc-9836-64b0f1ec1a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.554247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4720c5-50a1-48dc-9836-64b0f1ec1a72 admin admin] PATH_INFO: `/v3/projects/ca706bc9087142fdb0941f08e782d7d0/users/45c73ab2c2874e0286214cf1588aad50/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.559417 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 641/1285] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:34.562883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e13c7e-e491-4943-9e42-8b207a3ae8ae None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.562883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e13c7e-e491-4943-9e42-8b207a3ae8ae None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.562883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e13c7e-e491-4943-9e42-8b207a3ae8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.571661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4720c5-50a1-48dc-9836-64b0f1ec1a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca706bc9087142fdb0941f08e782d7d0, user_id=45c73ab2c2874e0286214cf1588aad50, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:34.575076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4720c5-50a1-48dc-9836-64b0f1ec1a72 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:34.599993 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 645/1286] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:34 2022] PUT /identity/v3/projects/ca706bc9087142fdb0941f08e782d7d0/users/45c73ab2c2874e0286214cf1588aad50/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:34.607059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cdfc0-b0ac-4151-b840-203375f92ef7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.607059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cdfc0-b0ac-4151-b840-203375f92ef7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.607059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cdfc0-b0ac-4151-b840-203375f92ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.608419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c4e13c7e-e491-4943-9e42-8b207a3ae8ae None None] MFA Rules not processed for user `678ce12a88374b1599c5bb89075a68cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:34.608419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c4e13c7e-e491-4943-9e42-8b207a3ae8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:34.653525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7b2cdfc0-b0ac-4151-b840-203375f92ef7 None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:34.654761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7b2cdfc0-b0ac-4151-b840-203375f92ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:34.704621 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 642/1287] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:34.705740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e13c7e-e491-4943-9e42-8b207a3ae8ae None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:34.707863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c4e13c7e-e491-4943-9e42-8b207a3ae8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:34.804841 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 646/1288] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Oct 24 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:34.804841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cd1e8-2bcd-447e-9700-d5437e974aed None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.804841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cd1e8-2bcd-447e-9700-d5437e974aed None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.804841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cd1e8-2bcd-447e-9700-d5437e974aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.804841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-823cd1e8-2bcd-447e-9700-d5437e974aed None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:34.804841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-823cd1e8-2bcd-447e-9700-d5437e974aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:34.813968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e13c7e-e491-4943-9e42-8b207a3ae8ae None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:34.820923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f732f-2153-4e71-b473-8dc7722b425c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:34.822069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f732f-2153-4e71-b473-8dc7722b425c service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.822593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f732f-2153-4e71-b473-8dc7722b425c service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.823128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f732f-2153-4e71-b473-8dc7722b425c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.890592 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 647/1289] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:34.892870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7581b98-e54d-489d-8696-a4e092830816 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.893163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7581b98-e54d-489d-8696-a4e092830816 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.893580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7581b98-e54d-489d-8696-a4e092830816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.915455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b7581b98-e54d-489d-8696-a4e092830816 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:34.916819 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b7581b98-e54d-489d-8696-a4e092830816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:34.916944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f732f-2153-4e71-b473-8dc7722b425c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:34.918951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f732f-2153-4e71-b473-8dc7722b425c service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:34.928140 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 648/1290] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:34.929806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7581b98-e54d-489d-8696-a4e092830816 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:34.938100 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 643/1291] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:34.939468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54785159-f990-4134-a857-1ff15a03245d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.939758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54785159-f990-4134-a857-1ff15a03245d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.940029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54785159-f990-4134-a857-1ff15a03245d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.945340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7581b98-e54d-489d-8696-a4e092830816 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:34.949065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a034665-5760-4b82-a3f8-b1e7d1c7774a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:34.949764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a034665-5760-4b82-a3f8-b1e7d1c7774a service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:34.950107 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a034665-5760-4b82-a3f8-b1e7d1c7774a service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:34.950452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a034665-5760-4b82-a3f8-b1e7d1c7774a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:34.963158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a034665-5760-4b82-a3f8-b1e7d1c7774a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:34.964717 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a034665-5760-4b82-a3f8-b1e7d1c7774a service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:34.976750 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-54785159-f990-4134-a857-1ff15a03245d None None] MFA Rules not processed for user `45c73ab2c2874e0286214cf1588aad50`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:34.978196 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 649/1292] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:34.978935 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-54785159-f990-4134-a857-1ff15a03245d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:34.979222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a034665-5760-4b82-a3f8-b1e7d1c7774a service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:34.989336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:34.997891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:35.059916 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a034665-5760-4b82-a3f8-b1e7d1c7774a service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:35.063812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e882168-395c-4fcd-b487-de1874988cb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:35.064562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882168-395c-4fcd-b487-de1874988cb7 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.064840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882168-395c-4fcd-b487-de1874988cb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.065111 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882168-395c-4fcd-b487-de1874988cb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.076859 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e882168-395c-4fcd-b487-de1874988cb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:35.079115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e882168-395c-4fcd-b487-de1874988cb7 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:35.086456 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 644/1293] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:35.088490 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54785159-f990-4134-a857-1ff15a03245d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:35.092104 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 650/1294] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:35.093038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e882168-395c-4fcd-b487-de1874988cb7 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:35.107463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e882168-395c-4fcd-b487-de1874988cb7 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:35.111555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54785159-f990-4134-a857-1ff15a03245d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:35.112844 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df33e4-c69a-463e-9b02-8ecfaaacae0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:35.113578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df33e4-c69a-463e-9b02-8ecfaaacae0c service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.113954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df33e4-c69a-463e-9b02-8ecfaaacae0c service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.114378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df33e4-c69a-463e-9b02-8ecfaaacae0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.130915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df33e4-c69a-463e-9b02-8ecfaaacae0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:35.132852 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df33e4-c69a-463e-9b02-8ecfaaacae0c service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:35.135470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749ca5f-e65d-4b23-ac83-dc2a9b963eca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:35.136314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749ca5f-e65d-4b23-ac83-dc2a9b963eca service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.136796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749ca5f-e65d-4b23-ac83-dc2a9b963eca service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.137070 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749ca5f-e65d-4b23-ac83-dc2a9b963eca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.147452 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 651/1295] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:35.148831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df33e4-c69a-463e-9b02-8ecfaaacae0c service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:35.160619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df33e4-c69a-463e-9b02-8ecfaaacae0c service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:35.167397 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a11e4-37a7-4a47-9576-e094b4499a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:35.168560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a11e4-37a7-4a47-9576-e094b4499a76 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.168831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a11e4-37a7-4a47-9576-e094b4499a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.169110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a11e4-37a7-4a47-9576-e094b4499a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.183166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a11e4-37a7-4a47-9576-e094b4499a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:35.184907 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a11e4-37a7-4a47-9576-e094b4499a76 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:35.203698 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 652/1296] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:35.204668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a11e4-37a7-4a47-9576-e094b4499a76 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:35.221918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a11e4-37a7-4a47-9576-e094b4499a76 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:35.224526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749ca5f-e65d-4b23-ac83-dc2a9b963eca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:35.226797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749ca5f-e65d-4b23-ac83-dc2a9b963eca service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:35.227809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02153e08-029e-47cc-90c5-40d20bc1c3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:35.228432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02153e08-029e-47cc-90c5-40d20bc1c3aa admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.228707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02153e08-029e-47cc-90c5-40d20bc1c3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.228982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02153e08-029e-47cc-90c5-40d20bc1c3aa admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.229672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02153e08-029e-47cc-90c5-40d20bc1c3aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:35.233714 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02153e08-029e-47cc-90c5-40d20bc1c3aa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:35.237857 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 645/1297] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:35.282333 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 653/1298] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:35 2022] POST /identity/v3/projects => generated 355 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:35.289388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749ca5f-e65d-4b23-ac83-dc2a9b963eca service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:35.310113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749ca5f-e65d-4b23-ac83-dc2a9b963eca service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:35.473691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0493b21-b7fc-40ca-b93f-aba4ee96c210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:35.474422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0493b21-b7fc-40ca-b93f-aba4ee96c210 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.474708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0493b21-b7fc-40ca-b93f-aba4ee96c210 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.475014 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0493b21-b7fc-40ca-b93f-aba4ee96c210 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.475781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0493b21-b7fc-40ca-b93f-aba4ee96c210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:35.481684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0493b21-b7fc-40ca-b93f-aba4ee96c210 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:35.527160 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 646/1299] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:35 2022] POST /identity/v3/users => generated 284 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:35.528117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02153e08-029e-47cc-90c5-40d20bc1c3aa admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:35.547928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02153e08-029e-47cc-90c5-40d20bc1c3aa admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:35.567514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afbca0-08e9-4689-9721-45df0bf3a099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:35.568177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afbca0-08e9-4689-9721-45df0bf3a099 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.570292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afbca0-08e9-4689-9721-45df0bf3a099 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.570436 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afbca0-08e9-4689-9721-45df0bf3a099 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.572774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afbca0-08e9-4689-9721-45df0bf3a099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:35.577752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afbca0-08e9-4689-9721-45df0bf3a099 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:35.605112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0493b21-b7fc-40ca-b93f-aba4ee96c210 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:35.605890 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 654/1300] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:35.618555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0493b21-b7fc-40ca-b93f-aba4ee96c210 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:35.636325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bd7a8-dccd-46a4-ab96-030d028a8015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:35.637008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bd7a8-dccd-46a4-ab96-030d028a8015 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.637289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bd7a8-dccd-46a4-ab96-030d028a8015 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.637566 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bd7a8-dccd-46a4-ab96-030d028a8015 admin admin] PATH_INFO: `/v3/projects/4f5cd4ccdd0d4f81b280c1d9ed96e359/users/9cb17b317a1d4aeb82570db8acb0fa61/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.686916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bd7a8-dccd-46a4-ab96-030d028a8015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5cd4ccdd0d4f81b280c1d9ed96e359, user_id=9cb17b317a1d4aeb82570db8acb0fa61, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:35.691352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bd7a8-dccd-46a4-ab96-030d028a8015 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:35.727014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aee66-52bb-4e78-94b9-9f26450d015e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.727260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aee66-52bb-4e78-94b9-9f26450d015e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.727530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aee66-52bb-4e78-94b9-9f26450d015e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.729978 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 647/1301] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:35 2022] PUT /identity/v3/projects/4f5cd4ccdd0d4f81b280c1d9ed96e359/users/9cb17b317a1d4aeb82570db8acb0fa61/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:47:35.730214 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 655/1302] 10.222.0.26 () {54 vars in 912 bytes} [Mon Oct 24 08:47:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:47:35.734163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bd7a8-dccd-46a4-ab96-030d028a8015 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:35.743375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6a65f-7def-4155-b160-4d8116ce2c8b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.743651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6a65f-7def-4155-b160-4d8116ce2c8b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.743923 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6a65f-7def-4155-b160-4d8116ce2c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.780719 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:35.783598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-47e6a65f-7def-4155-b160-4d8116ce2c8b None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:35.784856 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-47e6a65f-7def-4155-b160-4d8116ce2c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:35.924647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bd7a8-dccd-46a4-ab96-030d028a8015 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:35.945042 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ef25-3822-4e2e-877b-cc8c825dd07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:35.947849 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ef25-3822-4e2e-877b-cc8c825dd07b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:35.948179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ef25-3822-4e2e-877b-cc8c825dd07b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:35.948442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ef25-3822-4e2e-877b-cc8c825dd07b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:35.951716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77ef25-3822-4e2e-877b-cc8c825dd07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:35.955747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77ef25-3822-4e2e-877b-cc8c825dd07b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:35.965625 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 656/1303] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:47:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:35.975447 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 648/1304] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:35.979836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6a65f-7def-4155-b160-4d8116ce2c8b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:35.993426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ef25-3822-4e2e-877b-cc8c825dd07b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:36.001425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6a65f-7def-4155-b160-4d8116ce2c8b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:36.014871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fd297-12d9-4441-b2a7-d684e47719f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:36.015553 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fd297-12d9-4441-b2a7-d684e47719f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:36.015832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fd297-12d9-4441-b2a7-d684e47719f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:36.016102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fd297-12d9-4441-b2a7-d684e47719f4 admin admin] PATH_INFO: `/v3/projects/4f5cd4ccdd0d4f81b280c1d9ed96e359/users/9cb17b317a1d4aeb82570db8acb0fa61/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:36.021408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fd297-12d9-4441-b2a7-d684e47719f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5cd4ccdd0d4f81b280c1d9ed96e359, user_id=9cb17b317a1d4aeb82570db8acb0fa61, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:36.024233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fd297-12d9-4441-b2a7-d684e47719f4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:36.055883 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 657/1305] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:35 2022] PUT /identity/v3/projects/4f5cd4ccdd0d4f81b280c1d9ed96e359/users/9cb17b317a1d4aeb82570db8acb0fa61/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:36.066246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe0e83-11d2-491f-9ada-515c80d90b73 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:36.066592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe0e83-11d2-491f-9ada-515c80d90b73 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:36.066857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe0e83-11d2-491f-9ada-515c80d90b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:36.121584 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-debe0e83-11d2-491f-9ada-515c80d90b73 None None] MFA Rules not processed for user `9cb17b317a1d4aeb82570db8acb0fa61`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:36.122821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-debe0e83-11d2-491f-9ada-515c80d90b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:36.143545 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ef25-3822-4e2e-877b-cc8c825dd07b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:36.174352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:36.290845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e82b8-5fa5-46f8-9fbd-a42a48cda4f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:36.293118 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e82b8-5fa5-46f8-9fbd-a42a48cda4f3 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:36.296916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e82b8-5fa5-46f8-9fbd-a42a48cda4f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:36.298585 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e82b8-5fa5-46f8-9fbd-a42a48cda4f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:36.361195 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 658/1306] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:47:36.374951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe0e83-11d2-491f-9ada-515c80d90b73 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:36.394362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe0e83-11d2-491f-9ada-515c80d90b73 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:36.402785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f0643-1afd-41e9-b0e2-5350e2b77365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:36.403654 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f0643-1afd-41e9-b0e2-5350e2b77365 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:36.404102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f0643-1afd-41e9-b0e2-5350e2b77365 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:36.404704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f0643-1afd-41e9-b0e2-5350e2b77365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:36.431279 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f0643-1afd-41e9-b0e2-5350e2b77365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:36.433208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f0643-1afd-41e9-b0e2-5350e2b77365 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:36.460412 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 659/1307] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:36.564485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e82b8-5fa5-46f8-9fbd-a42a48cda4f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:36.572157 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e82b8-5fa5-46f8-9fbd-a42a48cda4f3 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:36.636825 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 649/1308] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:36.743626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f0643-1afd-41e9-b0e2-5350e2b77365 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:36.754258 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f0643-1afd-41e9-b0e2-5350e2b77365 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:36.771696 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd59276-814a-4409-8726-3d530b3272b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:36.772381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd59276-814a-4409-8726-3d530b3272b1 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:36.772663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd59276-814a-4409-8726-3d530b3272b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:36.773069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd59276-814a-4409-8726-3d530b3272b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:36.929037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd59276-814a-4409-8726-3d530b3272b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:36.933624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd59276-814a-4409-8726-3d530b3272b1 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:36.959733 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 660/1309] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:37.876891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67531-f9d3-4d57-b206-d80869a33aa4 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:37.877606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67531-f9d3-4d57-b206-d80869a33aa4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:37.878335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67531-f9d3-4d57-b206-d80869a33aa4 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:37.879992 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 650/1310] 10.222.0.26 () {54 vars in 961 bytes} [Mon Oct 24 08:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:47:37.889273 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec914e21-8941-4e4b-bf6b-f0541b9e416a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:37.889919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec914e21-8941-4e4b-bf6b-f0541b9e416a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:37.890043 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec914e21-8941-4e4b-bf6b-f0541b9e416a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:37.927455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ec914e21-8941-4e4b-bf6b-f0541b9e416a None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:37.932927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ec914e21-8941-4e4b-bf6b-f0541b9e416a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:37.961491 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 661/1311] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Oct 24 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:37.968061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45add53-bbb1-4e3a-959c-c131e842485e None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:37.968631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45add53-bbb1-4e3a-959c-c131e842485e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:37.969385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45add53-bbb1-4e3a-959c-c131e842485e None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:37.972592 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 651/1312] 10.222.0.26 () {54 vars in 912 bytes} [Mon Oct 24 08:47:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:47:37.979283 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b2d14-7df1-4178-80da-56b196f66ab8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:37.980783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b2d14-7df1-4178-80da-56b196f66ab8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:37.981316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b2d14-7df1-4178-80da-56b196f66ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:38.050991 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fb1b2d14-7df1-4178-80da-56b196f66ab8 None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:38.059287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fb1b2d14-7df1-4178-80da-56b196f66ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:38.086252 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 662/1313] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:38.100421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45add53-bbb1-4e3a-959c-c131e842485e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:38.118449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45add53-bbb1-4e3a-959c-c131e842485e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:38.130531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb236bf-39f9-4087-9deb-726b88e0e2ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:38.131135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb236bf-39f9-4087-9deb-726b88e0e2ce service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:38.131442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb236bf-39f9-4087-9deb-726b88e0e2ce service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:38.133964 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb236bf-39f9-4087-9deb-726b88e0e2ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:38.160537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb236bf-39f9-4087-9deb-726b88e0e2ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:38.162585 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb236bf-39f9-4087-9deb-726b88e0e2ce service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:38.169437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:38.180731 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 652/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:40.048061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918f19-f60c-417a-b3a2-1cc6f425c848 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:40.048563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918f19-f60c-417a-b3a2-1cc6f425c848 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:40.049418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918f19-f60c-417a-b3a2-1cc6f425c848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:40.114502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb236bf-39f9-4087-9deb-726b88e0e2ce service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:40.121189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f5918f19-f60c-417a-b3a2-1cc6f425c848 None None] MFA Rules not processed for user `7d793a1f26a3467ea8119e27e3394371`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:40.122442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f5918f19-f60c-417a-b3a2-1cc6f425c848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:40.231209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb236bf-39f9-4087-9deb-726b88e0e2ce service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:40.241353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:40.242007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:40.242284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:40.242557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] PATH_INFO: `/v3/users/c300409c40db4636817fd34bfa7c4c77` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:40.247278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c300409c40db4636817fd34bfa7c4c77)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:40.249901 dv-cinder-master-3-smb devstack@keystone.service[87872]: WARNING py.warnings [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:47:40.249901 dv-cinder-master-3-smb devstack@keystone.service[87872]: warnings.warn(msg) Oct 24 08:47:40.249901 dv-cinder-master-3-smb devstack@keystone.service[87872]: Oct 24 08:47:40.251178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:40.258012 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 663/1315] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:40.275885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918f19-f60c-417a-b3a2-1cc6f425c848 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:40.312263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918f19-f60c-417a-b3a2-1cc6f425c848 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:40.321431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e68b19-33b5-4600-815f-ac76bc88579b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:40.322810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e68b19-33b5-4600-815f-ac76bc88579b service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:40.323450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e68b19-33b5-4600-815f-ac76bc88579b service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:40.324243 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e68b19-33b5-4600-815f-ac76bc88579b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:40.344020 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e68b19-33b5-4600-815f-ac76bc88579b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:40.346934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e68b19-33b5-4600-815f-ac76bc88579b service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:40.364738 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 664/1316] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:40.416486 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c300409c40db4636817fd34bfa7c4c77'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:40.417169 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c300409c40db4636817fd34bfa7c4c77'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:40.438502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c300409c40db4636817fd34bfa7c4c77'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:40.457067 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c300409c40db4636817fd34bfa7c4c77'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:40.495283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c300409c40db4636817fd34bfa7c4c77'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:40.514053 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 653/1317] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:47:40 2022] DELETE /identity/v3/users/c300409c40db4636817fd34bfa7c4c77 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:47:40.515927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88ae0e-f8f3-40f7-86bb-b0571997174e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:40.517162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88ae0e-f8f3-40f7-86bb-b0571997174e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:40.518049 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88ae0e-f8f3-40f7-86bb-b0571997174e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:40.529045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:40.578259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0c88ae0e-f8f3-40f7-86bb-b0571997174e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:40.579623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0c88ae0e-f8f3-40f7-86bb-b0571997174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:40.658132 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 665/1318] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:40.997345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:40.999477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:41.210471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e2df6-35de-4b83-88ba-09e1de915ec4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:41.233082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91e6d2-0b24-4469-bcbb-9cc92c7a2a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:41.233082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91e6d2-0b24-4469-bcbb-9cc92c7a2a8d service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:41.233082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91e6d2-0b24-4469-bcbb-9cc92c7a2a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:41.233082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91e6d2-0b24-4469-bcbb-9cc92c7a2a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:41.255338 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91e6d2-0b24-4469-bcbb-9cc92c7a2a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:41.256788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91e6d2-0b24-4469-bcbb-9cc92c7a2a8d service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:41.296904 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 654/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:43.003553 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88ae0e-f8f3-40f7-86bb-b0571997174e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:43.005683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0c88ae0e-f8f3-40f7-86bb-b0571997174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:43.023013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88ae0e-f8f3-40f7-86bb-b0571997174e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:43.049433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:43.050492 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:43.050826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:43.051151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] PATH_INFO: `/v3/projects/35e1a122b4354e2880b363d6f3815278` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:43.052510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e1a122b4354e2880b363d6f3815278)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:43.053608 dv-cinder-master-3-smb devstack@keystone.service[87873]: WARNING py.warnings [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 08:47:43.053608 dv-cinder-master-3-smb devstack@keystone.service[87873]: warnings.warn(msg) Oct 24 08:47:43.053608 dv-cinder-master-3-smb devstack@keystone.service[87873]: Oct 24 08:47:43.055094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:43.174583 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e1a122b4354e2880b363d6f3815278', 'request_id': 'req-6cdeb6e6-104f-47cf-ab49-001d514be758'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:43.192001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e1a122b4354e2880b363d6f3815278', 'request_id': 'req-6cdeb6e6-104f-47cf-ab49-001d514be758'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:43.230215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e1a122b4354e2880b363d6f3815278', 'request_id': 'req-6cdeb6e6-104f-47cf-ab49-001d514be758'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:43.254081 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] The token cache is being invalidate because project 35e1a122b4354e2880b363d6f3815278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:47:43.257599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:43.261300 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cdeb6e6-104f-47cf-ab49-001d514be758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:43.268375 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 666/1320] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:47:43 2022] DELETE /identity/v3/projects/35e1a122b4354e2880b363d6f3815278 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:47:43.273750 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b2dbf-f1ff-483c-8bf4-49891c88691c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:43.274261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b2dbf-f1ff-483c-8bf4-49891c88691c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:43.274503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b2dbf-f1ff-483c-8bf4-49891c88691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:43.311790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8b7b2dbf-f1ff-483c-8bf4-49891c88691c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:43.315437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8b7b2dbf-f1ff-483c-8bf4-49891c88691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:43.354026 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:43.495161 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 655/1321] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:43.503216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad460c3-29ab-4067-9e67-bda599879998 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:43.503763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad460c3-29ab-4067-9e67-bda599879998 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:43.504129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad460c3-29ab-4067-9e67-bda599879998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:43.550250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8ad460c3-29ab-4067-9e67-bda599879998 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:43.551689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8ad460c3-29ab-4067-9e67-bda599879998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:43.584274 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 667/1322] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:43.588728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b2dbf-f1ff-483c-8bf4-49891c88691c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:43.606503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b2dbf-f1ff-483c-8bf4-49891c88691c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:43.627092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715901c1-3f3f-4dcc-bb01-f7cdb936f684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:43.627984 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715901c1-3f3f-4dcc-bb01-f7cdb936f684 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:43.628779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715901c1-3f3f-4dcc-bb01-f7cdb936f684 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:43.629154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715901c1-3f3f-4dcc-bb01-f7cdb936f684 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:43.629932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715901c1-3f3f-4dcc-bb01-f7cdb936f684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:43.630336 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715901c1-3f3f-4dcc-bb01-f7cdb936f684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:43.632083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715901c1-3f3f-4dcc-bb01-f7cdb936f684 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:43.658442 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 656/1323] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:43.662937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad460c3-29ab-4067-9e67-bda599879998 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:43.674083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad460c3-29ab-4067-9e67-bda599879998 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:43.686759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1cf32-de54-47b8-a012-cbacd45b8ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:43.687571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1cf32-de54-47b8-a012-cbacd45b8ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:43.687945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1cf32-de54-47b8-a012-cbacd45b8ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:43.688313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1cf32-de54-47b8-a012-cbacd45b8ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:43.689072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1cf32-de54-47b8-a012-cbacd45b8ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:43.693068 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1cf32-de54-47b8-a012-cbacd45b8ed7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:43.764290 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 668/1324] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:43 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:47:43.768128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715901c1-3f3f-4dcc-bb01-f7cdb936f684 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:43.783215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715901c1-3f3f-4dcc-bb01-f7cdb936f684 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:43.932321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ea703-0122-48aa-b889-1d2f948c0e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:43.933064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ea703-0122-48aa-b889-1d2f948c0e56 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:43.933468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ea703-0122-48aa-b889-1d2f948c0e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:43.933812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ea703-0122-48aa-b889-1d2f948c0e56 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:43.934564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ea703-0122-48aa-b889-1d2f948c0e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:43.942022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ea703-0122-48aa-b889-1d2f948c0e56 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:43.990532 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 657/1325] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:43 2022] POST /identity/v3/users => generated 303 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:43.994474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1cf32-de54-47b8-a012-cbacd45b8ed7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:44.005824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1cf32-de54-47b8-a012-cbacd45b8ed7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:44.032307 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ff2e6-dce7-496b-8f9e-abfa7faf698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:44.037343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ff2e6-dce7-496b-8f9e-abfa7faf698a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:44.037771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ff2e6-dce7-496b-8f9e-abfa7faf698a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:44.038151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ff2e6-dce7-496b-8f9e-abfa7faf698a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:44.038967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ff2e6-dce7-496b-8f9e-abfa7faf698a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:44.047409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ff2e6-dce7-496b-8f9e-abfa7faf698a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:44.066785 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 669/1326] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:44.070553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ea703-0122-48aa-b889-1d2f948c0e56 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:44.083691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ea703-0122-48aa-b889-1d2f948c0e56 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:44.098926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00938d0-e535-4120-aa54-e7ee59941b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:44.099648 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00938d0-e535-4120-aa54-e7ee59941b99 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:44.100032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00938d0-e535-4120-aa54-e7ee59941b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:44.100409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00938d0-e535-4120-aa54-e7ee59941b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:44.101171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00938d0-e535-4120-aa54-e7ee59941b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:44.102936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00938d0-e535-4120-aa54-e7ee59941b99 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:44.134551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ff2e6-dce7-496b-8f9e-abfa7faf698a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:44.145625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ff2e6-dce7-496b-8f9e-abfa7faf698a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:44.145836 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 658/1327] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:44.157555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fead51-fbfe-4632-934d-42f032814b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:44.159382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fead51-fbfe-4632-934d-42f032814b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:44.159822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fead51-fbfe-4632-934d-42f032814b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:44.162112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fead51-fbfe-4632-934d-42f032814b7c admin admin] PATH_INFO: `/v3/projects/f9c650393b994a0ba7b3576021c21ea2/users/9a0c304691e948ee9e8e462dae7ddc2c/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:44.197719 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fead51-fbfe-4632-934d-42f032814b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c650393b994a0ba7b3576021c21ea2, user_id=9a0c304691e948ee9e8e462dae7ddc2c, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:44.203026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fead51-fbfe-4632-934d-42f032814b7c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:44.235165 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 670/1328] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:44 2022] PUT /identity/v3/projects/f9c650393b994a0ba7b3576021c21ea2/users/9a0c304691e948ee9e8e462dae7ddc2c/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:47:44.242076 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db054a-65ed-43fc-9f46-fa7f1c58a72e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:44.243897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db054a-65ed-43fc-9f46-fa7f1c58a72e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:44.244417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db054a-65ed-43fc-9f46-fa7f1c58a72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:44.303999 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-69db054a-65ed-43fc-9f46-fa7f1c58a72e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:44.305437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-69db054a-65ed-43fc-9f46-fa7f1c58a72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:44.506182 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 659/1329] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:44.542964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fead51-fbfe-4632-934d-42f032814b7c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:44.544805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e1fead51-fbfe-4632-934d-42f032814b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:44.593444 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:44.655254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fead51-fbfe-4632-934d-42f032814b7c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:44.665378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055897d9-153d-47bd-a506-85de1dfa3c96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:44.666091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055897d9-153d-47bd-a506-85de1dfa3c96 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:44.666930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055897d9-153d-47bd-a506-85de1dfa3c96 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:44.668456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055897d9-153d-47bd-a506-85de1dfa3c96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:44.687028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055897d9-153d-47bd-a506-85de1dfa3c96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:44.688741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055897d9-153d-47bd-a506-85de1dfa3c96 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:44.701447 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 671/1330] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:47.035889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016295-52d4-403d-b5df-90c2755380eb None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:47.036450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016295-52d4-403d-b5df-90c2755380eb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:47.036450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016295-52d4-403d-b5df-90c2755380eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:47.081422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-08016295-52d4-403d-b5df-90c2755380eb None None] MFA Rules not processed for user `7d793a1f26a3467ea8119e27e3394371`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:47.082663 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-08016295-52d4-403d-b5df-90c2755380eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:47.155165 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:47.262906 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 660/1331] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:48.758862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055897d9-153d-47bd-a506-85de1dfa3c96 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:48.760398 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-055897d9-153d-47bd-a506-85de1dfa3c96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:48.793193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055897d9-153d-47bd-a506-85de1dfa3c96 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:48.799817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:48.800563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:48.800957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:48.801342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] PATH_INFO: `/v3/users/678ce12a88374b1599c5bb89075a68cf` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:48.804736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=678ce12a88374b1599c5bb89075a68cf)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:48.806850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:48.900315 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '678ce12a88374b1599c5bb89075a68cf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:48.909845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '678ce12a88374b1599c5bb89075a68cf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:48.910910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '678ce12a88374b1599c5bb89075a68cf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:48.926452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '678ce12a88374b1599c5bb89075a68cf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:48.958701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '678ce12a88374b1599c5bb89075a68cf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:48.976948 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 672/1332] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:47:48 2022] DELETE /identity/v3/users/678ce12a88374b1599c5bb89075a68cf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:48.979014 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9459cd-4c89-4988-bf47-aacd737fffc5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:48.980629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9459cd-4c89-4988-bf47-aacd737fffc5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:48.981163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9459cd-4c89-4988-bf47-aacd737fffc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:49.038458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-dc9459cd-4c89-4988-bf47-aacd737fffc5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:49.040385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dc9459cd-4c89-4988-bf47-aacd737fffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:49.146037 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 661/1333] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:49.659295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:49.661019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:49.808748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825b4b7-12ea-46c9-b254-77b27a95bfdc admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:49.816968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923d2b6-74b5-4962-bf4d-ea48b296da1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:49.817613 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923d2b6-74b5-4962-bf4d-ea48b296da1a service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:49.817893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923d2b6-74b5-4962-bf4d-ea48b296da1a service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:49.818167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923d2b6-74b5-4962-bf4d-ea48b296da1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:49.848192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923d2b6-74b5-4962-bf4d-ea48b296da1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:49.849703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923d2b6-74b5-4962-bf4d-ea48b296da1a service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:49.872204 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 673/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:50.600484 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c450e81-2bb4-423f-b039-9ee9f548cdac None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:50.601139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c450e81-2bb4-423f-b039-9ee9f548cdac None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:50.601139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c450e81-2bb4-423f-b039-9ee9f548cdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:50.695973 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0c450e81-2bb4-423f-b039-9ee9f548cdac None None] MFA Rules not processed for user `712e5cf4cc004918817ab1802b9c69d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:50.697661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0c450e81-2bb4-423f-b039-9ee9f548cdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:50.880391 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 662/1335] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:50.898027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923d2b6-74b5-4962-bf4d-ea48b296da1a service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:50.902151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9923d2b6-74b5-4962-bf4d-ea48b296da1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:50.981361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f3961-b509-4f32-8f67-794e61db1bb7 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:50.981678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f3961-b509-4f32-8f67-794e61db1bb7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:50.982062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f3961-b509-4f32-8f67-794e61db1bb7 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:50.986516 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 663/1336] 10.222.0.26 () {54 vars in 912 bytes} [Mon Oct 24 08:47:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:47:50.991691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2716fa-24e5-4b3d-a346-ccfbc526c234 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:50.992173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2716fa-24e5-4b3d-a346-ccfbc526c234 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:50.992617 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2716fa-24e5-4b3d-a346-ccfbc526c234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:51.044928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:51.127225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2f2716fa-24e5-4b3d-a346-ccfbc526c234 None None] MFA Rules not processed for user `2254623f523c48f0aa454b85be41df95`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:51.150682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2f2716fa-24e5-4b3d-a346-ccfbc526c234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:51.178780 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 664/1337] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:47:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:51.181157 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923d2b6-74b5-4962-bf4d-ea48b296da1a service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:51.182731 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ac01e-32b7-469b-9a07-f7159abcb8e6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:51.186004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ac01e-32b7-469b-9a07-f7159abcb8e6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:51.187318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ac01e-32b7-469b-9a07-f7159abcb8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:51.203118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80423ae9-c85a-4ae7-a080-ee8986c73bb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:51.204144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80423ae9-c85a-4ae7-a080-ee8986c73bb8 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:51.204642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80423ae9-c85a-4ae7-a080-ee8986c73bb8 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:51.207445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80423ae9-c85a-4ae7-a080-ee8986c73bb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:51.229649 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80423ae9-c85a-4ae7-a080-ee8986c73bb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:51.234890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80423ae9-c85a-4ae7-a080-ee8986c73bb8 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:51.261231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-084ac01e-32b7-469b-9a07-f7159abcb8e6 None None] MFA Rules not processed for user `9a0c304691e948ee9e8e462dae7ddc2c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:51.262474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-084ac01e-32b7-469b-9a07-f7159abcb8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:51.266659 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 674/1338] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:51.368223 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:51.476670 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 665/1339] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:51 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:51.481384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80423ae9-c85a-4ae7-a080-ee8986c73bb8 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:51.483075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-80423ae9-c85a-4ae7-a080-ee8986c73bb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:51.512310 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80423ae9-c85a-4ae7-a080-ee8986c73bb8 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:51.537257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e7045-f135-4f97-bc98-ce2715e4417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:51.537909 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e7045-f135-4f97-bc98-ce2715e4417b admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:51.538193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e7045-f135-4f97-bc98-ce2715e4417b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:51.538464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e7045-f135-4f97-bc98-ce2715e4417b admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:51.547129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e7045-f135-4f97-bc98-ce2715e4417b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:51.552452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e7045-f135-4f97-bc98-ce2715e4417b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:51.647673 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 675/1340] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:51 2022] POST /identity/v3/projects => generated 403 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:51.653066 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ac01e-32b7-469b-9a07-f7159abcb8e6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:51.676242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ac01e-32b7-469b-9a07-f7159abcb8e6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:51.807960 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e7045-f135-4f97-bc98-ce2715e4417b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:51.846291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165d40a-b276-42b3-8b60-3e0fec8cb5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:51.847088 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165d40a-b276-42b3-8b60-3e0fec8cb5fc admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:51.847519 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165d40a-b276-42b3-8b60-3e0fec8cb5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:51.847921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165d40a-b276-42b3-8b60-3e0fec8cb5fc admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:51.851307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165d40a-b276-42b3-8b60-3e0fec8cb5fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:51.853472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165d40a-b276-42b3-8b60-3e0fec8cb5fc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:51.915622 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 666/1341] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:51 2022] POST /identity/v3/users => generated 308 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:51.917299 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165d40a-b276-42b3-8b60-3e0fec8cb5fc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:51.935460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165d40a-b276-42b3-8b60-3e0fec8cb5fc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:51.945736 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b209fa-00bc-4373-8f3c-5d8d36fe3466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:51.948100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b209fa-00bc-4373-8f3c-5d8d36fe3466 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:51.948814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b209fa-00bc-4373-8f3c-5d8d36fe3466 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:51.950306 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b209fa-00bc-4373-8f3c-5d8d36fe3466 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:51.951124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b209fa-00bc-4373-8f3c-5d8d36fe3466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:51.953441 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e7045-f135-4f97-bc98-ce2715e4417b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:51.958307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b209fa-00bc-4373-8f3c-5d8d36fe3466 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:51.958480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dee5b-aaf6-4c9b-b1db-822ac62735f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:51.959325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dee5b-aaf6-4c9b-b1db-822ac62735f9 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:51.959668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dee5b-aaf6-4c9b-b1db-822ac62735f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:51.960109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dee5b-aaf6-4c9b-b1db-822ac62735f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:51.970367 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293dee5b-aaf6-4c9b-b1db-822ac62735f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:51.972368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293dee5b-aaf6-4c9b-b1db-822ac62735f9 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:51.977239 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 667/1342] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:51.981832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b209fa-00bc-4373-8f3c-5d8d36fe3466 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:51.984245 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 676/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:47:51.995559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b209fa-00bc-4373-8f3c-5d8d36fe3466 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:52.002426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5bff-7acc-44dc-af88-6a3e5b305c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:52.004001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5bff-7acc-44dc-af88-6a3e5b305c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:52.004363 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5bff-7acc-44dc-af88-6a3e5b305c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:52.004957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5bff-7acc-44dc-af88-6a3e5b305c66 admin admin] PATH_INFO: `/v3/projects/bdc465febb3d4e86b20f0ebd00795884/users/2b6b4f711c354d279cdb164c093d6a8b/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:52.026350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5bff-7acc-44dc-af88-6a3e5b305c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc465febb3d4e86b20f0ebd00795884, user_id=2b6b4f711c354d279cdb164c093d6a8b, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:52.030250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5bff-7acc-44dc-af88-6a3e5b305c66 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:52.078073 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 668/1344] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:51 2022] PUT /identity/v3/projects/bdc465febb3d4e86b20f0ebd00795884/users/2b6b4f711c354d279cdb164c093d6a8b/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:52.080979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dee5b-aaf6-4c9b-b1db-822ac62735f9 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:52.091480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dee5b-aaf6-4c9b-b1db-822ac62735f9 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:52.207269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f4109-900d-4189-a010-766710f9ac42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:52.208825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f4109-900d-4189-a010-766710f9ac42 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:52.209314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f4109-900d-4189-a010-766710f9ac42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:52.210009 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f4109-900d-4189-a010-766710f9ac42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:52.210922 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f4109-900d-4189-a010-766710f9ac42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:52.214744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f4109-900d-4189-a010-766710f9ac42 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:52.232867 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 677/1345] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:47:52.236320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5bff-7acc-44dc-af88-6a3e5b305c66 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:52.255600 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5bff-7acc-44dc-af88-6a3e5b305c66 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:52.266173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16a5d5-9933-429b-a565-3c47f56a1b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:52.268112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16a5d5-9933-429b-a565-3c47f56a1b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:52.269426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16a5d5-9933-429b-a565-3c47f56a1b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:52.270927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16a5d5-9933-429b-a565-3c47f56a1b50 admin admin] PATH_INFO: `/v3/projects/bdc465febb3d4e86b20f0ebd00795884/users/2b6b4f711c354d279cdb164c093d6a8b/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:52.286991 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16a5d5-9933-429b-a565-3c47f56a1b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc465febb3d4e86b20f0ebd00795884, user_id=2b6b4f711c354d279cdb164c093d6a8b, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:52.295951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16a5d5-9933-429b-a565-3c47f56a1b50 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:52.314570 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 669/1346] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:52 2022] PUT /identity/v3/projects/bdc465febb3d4e86b20f0ebd00795884/users/2b6b4f711c354d279cdb164c093d6a8b/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:52.782649 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f4109-900d-4189-a010-766710f9ac42 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:52.787134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ec4f4109-900d-4189-a010-766710f9ac42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:52.848256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f4109-900d-4189-a010-766710f9ac42 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:52.969670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:53.013017 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:53.014870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:53.015408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:53.015818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] PATH_INFO: `/v3/projects/fd5ce90003ca44b69bb7f58333ca1f71` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:53.017795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd5ce90003ca44b69bb7f58333ca1f71)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:53.027680 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:53.140076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd5ce90003ca44b69bb7f58333ca1f71', 'request_id': 'req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:53.151830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace5974-b9de-4850-9f76-64a09d548485 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:53.152317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace5974-b9de-4850-9f76-64a09d548485 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:53.152763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace5974-b9de-4850-9f76-64a09d548485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:53.179686 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd5ce90003ca44b69bb7f58333ca1f71', 'request_id': 'req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:53.227400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd5ce90003ca44b69bb7f58333ca1f71', 'request_id': 'req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:53.254372 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] The token cache is being invalidate because project fd5ce90003ca44b69bb7f58333ca1f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:47:53.254633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:53.255478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b2646e6-8448-4e13-b85c-6d5b2bf2aa4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:53.256522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8ace5974-b9de-4850-9f76-64a09d548485 None None] MFA Rules not processed for user `b002748b80db40358571017d6b42ed09`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:53.261462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8ace5974-b9de-4850-9f76-64a09d548485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:53.261605 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 678/1347] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:47:52 2022] DELETE /identity/v3/projects/fd5ce90003ca44b69bb7f58333ca1f71 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:53.263160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c3915-2680-4c1e-ad91-3c246de278f1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:53.263445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c3915-2680-4c1e-ad91-3c246de278f1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:53.263789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c3915-2680-4c1e-ad91-3c246de278f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:53.315766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e85c3915-2680-4c1e-ad91-3c246de278f1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:53.318266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e85c3915-2680-4c1e-ad91-3c246de278f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:53.477024 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 670/1348] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:53.488806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f6cbb-8c05-433a-806b-8bf6060c5c08 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:53.489664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f6cbb-8c05-433a-806b-8bf6060c5c08 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:53.490305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f6cbb-8c05-433a-806b-8bf6060c5c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:53.520326 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:53.570022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bf7f6cbb-8c05-433a-806b-8bf6060c5c08 None None] MFA Rules not processed for user `b002748b80db40358571017d6b42ed09`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:53.573767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bf7f6cbb-8c05-433a-806b-8bf6060c5c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:53.611061 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 671/1349] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:47:53.612446 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f6cbb-8c05-433a-806b-8bf6060c5c08 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:53.618916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bf7f6cbb-8c05-433a-806b-8bf6060c5c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:53.650899 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 679/1350] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:53.662851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c3915-2680-4c1e-ad91-3c246de278f1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:53.666655 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e85c3915-2680-4c1e-ad91-3c246de278f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:53.783178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f6cbb-8c05-433a-806b-8bf6060c5c08 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:53.795541 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707b70d-5111-41c3-bb1f-efd4bc3de998 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:53.800505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b70d-5111-41c3-bb1f-efd4bc3de998 service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:53.801179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b70d-5111-41c3-bb1f-efd4bc3de998 service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:53.801558 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707b70d-5111-41c3-bb1f-efd4bc3de998 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:53.807796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e707b70d-5111-41c3-bb1f-efd4bc3de998 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:53.838863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707b70d-5111-41c3-bb1f-efd4bc3de998 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:53.841037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707b70d-5111-41c3-bb1f-efd4bc3de998 service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:53.873006 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 672/1351] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:53.874732 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484536df-02ac-4b99-be30-2d1a3cd44d4d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:53.875708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484536df-02ac-4b99-be30-2d1a3cd44d4d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:53.879029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484536df-02ac-4b99-be30-2d1a3cd44d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:53.927504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c3915-2680-4c1e-ad91-3c246de278f1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:53.945619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28f51d-f4c2-4ac4-ae51-af108c0a7d34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:53.947756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28f51d-f4c2-4ac4-ae51-af108c0a7d34 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:53.947977 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-484536df-02ac-4b99-be30-2d1a3cd44d4d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:53.948338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28f51d-f4c2-4ac4-ae51-af108c0a7d34 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:53.949638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-484536df-02ac-4b99-be30-2d1a3cd44d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:53.950061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28f51d-f4c2-4ac4-ae51-af108c0a7d34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:53.967668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28f51d-f4c2-4ac4-ae51-af108c0a7d34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:53.969990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28f51d-f4c2-4ac4-ae51-af108c0a7d34 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:53.983089 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:53.987365 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 673/1352] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:53.988911 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484536df-02ac-4b99-be30-2d1a3cd44d4d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:54.001515 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 680/1353] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:54.039593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484536df-02ac-4b99-be30-2d1a3cd44d4d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:54.060441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa2211-5d45-4fd0-a115-84612225162a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:54.061268 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa2211-5d45-4fd0-a115-84612225162a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:54.061643 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa2211-5d45-4fd0-a115-84612225162a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:54.062611 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa2211-5d45-4fd0-a115-84612225162a admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:54.064709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa2211-5d45-4fd0-a115-84612225162a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:47:54.067322 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa2211-5d45-4fd0-a115-84612225162a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:54.069669 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa2211-5d45-4fd0-a115-84612225162a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:54.118105 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 674/1354] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:47:54.118214 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28f51d-f4c2-4ac4-ae51-af108c0a7d34 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:54.148058 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28f51d-f4c2-4ac4-ae51-af108c0a7d34 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:54.169989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d145e73-0cd7-454c-b9fd-9a761f198775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:54.170514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d145e73-0cd7-454c-b9fd-9a761f198775 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:54.175077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d145e73-0cd7-454c-b9fd-9a761f198775 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:54.180219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d145e73-0cd7-454c-b9fd-9a761f198775 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:54.183993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d145e73-0cd7-454c-b9fd-9a761f198775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:54.192483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d145e73-0cd7-454c-b9fd-9a761f198775 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:54.260912 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 681/1355] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:54 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:54.261922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa2211-5d45-4fd0-a115-84612225162a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:54.330152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa2211-5d45-4fd0-a115-84612225162a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:54.525779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26919293-e44d-400e-9eaf-ea94ae51d6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:54.530008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26919293-e44d-400e-9eaf-ea94ae51d6cd admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:54.530281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26919293-e44d-400e-9eaf-ea94ae51d6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:54.530571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26919293-e44d-400e-9eaf-ea94ae51d6cd admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:54.531289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26919293-e44d-400e-9eaf-ea94ae51d6cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:54.533276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26919293-e44d-400e-9eaf-ea94ae51d6cd admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:54.581383 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 675/1356] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:54 2022] POST /identity/v3/users => generated 283 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:54.587208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d145e73-0cd7-454c-b9fd-9a761f198775 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:54.606913 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d145e73-0cd7-454c-b9fd-9a761f198775 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:54.636383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd90e75-fa9d-4791-a70d-280782004c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:54.637052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd90e75-fa9d-4791-a70d-280782004c6c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:54.637330 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd90e75-fa9d-4791-a70d-280782004c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:54.637599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd90e75-fa9d-4791-a70d-280782004c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:54.638433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd90e75-fa9d-4791-a70d-280782004c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:54.641222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd90e75-fa9d-4791-a70d-280782004c6c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:54.682518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26919293-e44d-400e-9eaf-ea94ae51d6cd admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:54.684359 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 682/1357] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:54.705608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26919293-e44d-400e-9eaf-ea94ae51d6cd admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:54.744116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc52076-46a9-4a8a-8e4d-164db4a0da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:54.746388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc52076-46a9-4a8a-8e4d-164db4a0da04 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:54.746388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc52076-46a9-4a8a-8e4d-164db4a0da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:54.746388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc52076-46a9-4a8a-8e4d-164db4a0da04 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:54.746388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc52076-46a9-4a8a-8e4d-164db4a0da04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:54.748630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc52076-46a9-4a8a-8e4d-164db4a0da04 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:54.782323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd90e75-fa9d-4791-a70d-280782004c6c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:54.787561 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 676/1358] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:54.793862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd90e75-fa9d-4791-a70d-280782004c6c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:54.812298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a90c4-f4bf-4db7-ad20-7b0585f850f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:54.814917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a90c4-f4bf-4db7-ad20-7b0585f850f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:54.816197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a90c4-f4bf-4db7-ad20-7b0585f850f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:54.816481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a90c4-f4bf-4db7-ad20-7b0585f850f4 admin admin] PATH_INFO: `/v3/projects/44f6a6b8b0d24bbf88b3c320bdc21801/users/9fe3ac847852457db7f533067f2c1337/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:54.877443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a90c4-f4bf-4db7-ad20-7b0585f850f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f6a6b8b0d24bbf88b3c320bdc21801, user_id=9fe3ac847852457db7f533067f2c1337, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:54.894369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a90c4-f4bf-4db7-ad20-7b0585f850f4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:54.954892 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 683/1359] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:54 2022] PUT /identity/v3/projects/44f6a6b8b0d24bbf88b3c320bdc21801/users/9fe3ac847852457db7f533067f2c1337/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:47:54.964660 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c1bd9-84bd-42a9-8b4f-6479d8874e35 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:54.965096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c1bd9-84bd-42a9-8b4f-6479d8874e35 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:54.965379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c1bd9-84bd-42a9-8b4f-6479d8874e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:55.068643 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-571c1bd9-84bd-42a9-8b4f-6479d8874e35 None None] MFA Rules not processed for user `9fe3ac847852457db7f533067f2c1337`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:55.072154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-571c1bd9-84bd-42a9-8b4f-6479d8874e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:55.138371 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:55.237405 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 677/1360] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:55.243862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a90c4-f4bf-4db7-ad20-7b0585f850f4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:55.393833 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a90c4-f4bf-4db7-ad20-7b0585f850f4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:55.412037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48425f7-55d0-4b32-a221-8f03b0c022bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:55.412876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48425f7-55d0-4b32-a221-8f03b0c022bc admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:55.413420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48425f7-55d0-4b32-a221-8f03b0c022bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:55.413928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48425f7-55d0-4b32-a221-8f03b0c022bc admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:55.415174 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48425f7-55d0-4b32-a221-8f03b0c022bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:55.417503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48425f7-55d0-4b32-a221-8f03b0c022bc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:55.482794 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 684/1361] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:47:55 2022] POST /identity/v3/projects => generated 363 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:47:55.487266 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c1bd9-84bd-42a9-8b4f-6479d8874e35 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:55.527511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c1bd9-84bd-42a9-8b4f-6479d8874e35 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:55.708409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4ecd-83c7-4674-9b0b-3b05ac8cc563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:55.709475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4ecd-83c7-4674-9b0b-3b05ac8cc563 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:55.709761 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4ecd-83c7-4674-9b0b-3b05ac8cc563 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:55.710028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4ecd-83c7-4674-9b0b-3b05ac8cc563 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:55.710687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f4ecd-83c7-4674-9b0b-3b05ac8cc563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:55.713178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f4ecd-83c7-4674-9b0b-3b05ac8cc563 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:55.781340 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 678/1362] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:47:55 2022] POST /identity/v3/users => generated 288 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:47:55.785677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48425f7-55d0-4b32-a221-8f03b0c022bc admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:55.803989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48425f7-55d0-4b32-a221-8f03b0c022bc admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:55.811788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4ecd-83c7-4674-9b0b-3b05ac8cc563 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:55.813537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-031f4ecd-83c7-4674-9b0b-3b05ac8cc563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:55.832766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e37ac5-d937-41fa-a65a-925480e832e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:55.833412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e37ac5-d937-41fa-a65a-925480e832e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:55.833687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e37ac5-d937-41fa-a65a-925480e832e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:55.833958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e37ac5-d937-41fa-a65a-925480e832e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:55.834612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e37ac5-d937-41fa-a65a-925480e832e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:55.837471 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e37ac5-d937-41fa-a65a-925480e832e0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:55.842443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4ecd-83c7-4674-9b0b-3b05ac8cc563 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:55.862227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:55.862998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:55.863275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:55.875808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] PATH_INFO: `/v3/users/45c73ab2c2874e0286214cf1588aad50` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:55.877741 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c73ab2c2874e0286214cf1588aad50)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:55.880233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:55.881913 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 685/1363] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:47:55.883251 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e37ac5-d937-41fa-a65a-925480e832e0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:55.911217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e37ac5-d937-41fa-a65a-925480e832e0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:55.917739 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19589e-dc5d-47d0-9e67-a72089c9427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:55.921245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19589e-dc5d-47d0-9e67-a72089c9427c admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:55.922225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19589e-dc5d-47d0-9e67-a72089c9427c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:55.922694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19589e-dc5d-47d0-9e67-a72089c9427c admin admin] PATH_INFO: `/v3/projects/6cfd12ceb2a5454281d74296ac59f5c7/users/df70d29f9b35498c8e4704b59f98efbf/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:55.969043 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19589e-dc5d-47d0-9e67-a72089c9427c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfd12ceb2a5454281d74296ac59f5c7, user_id=df70d29f9b35498c8e4704b59f98efbf, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:55.975401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19589e-dc5d-47d0-9e67-a72089c9427c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:55.978872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c73ab2c2874e0286214cf1588aad50'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:55.979805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c73ab2c2874e0286214cf1588aad50'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:55.995368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c73ab2c2874e0286214cf1588aad50'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:56.002158 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 686/1364] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:55 2022] PUT /identity/v3/projects/6cfd12ceb2a5454281d74296ac59f5c7/users/df70d29f9b35498c8e4704b59f98efbf/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:56.005881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19589e-dc5d-47d0-9e67-a72089c9427c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:56.020894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c73ab2c2874e0286214cf1588aad50'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:56.051796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-79949517-977f-465f-b30d-3ce8ec0e4b7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c73ab2c2874e0286214cf1588aad50'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:56.067472 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 679/1365] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:47:55 2022] DELETE /identity/v3/users/45c73ab2c2874e0286214cf1588aad50 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:47:56.072220 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f74dd8-ad8a-4b5d-9de3-18b623212926 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:56.072506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f74dd8-ad8a-4b5d-9de3-18b623212926 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:56.072783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f74dd8-ad8a-4b5d-9de3-18b623212926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:56.120647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:56.124062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-95f74dd8-ad8a-4b5d-9de3-18b623212926 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:56.125414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-95f74dd8-ad8a-4b5d-9de3-18b623212926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:56.147309 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19589e-dc5d-47d0-9e67-a72089c9427c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:56.198232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:56.265055 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 680/1366] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:47:56.276008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b645d-d2b3-41d3-a40f-12c8b62b2734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:56.276812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b645d-d2b3-41d3-a40f-12c8b62b2734 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:56.277184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b645d-d2b3-41d3-a40f-12c8b62b2734 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:56.277546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b645d-d2b3-41d3-a40f-12c8b62b2734 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:56.278293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b645d-d2b3-41d3-a40f-12c8b62b2734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:56.280284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b645d-d2b3-41d3-a40f-12c8b62b2734 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:56.314950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f74dd8-ad8a-4b5d-9de3-18b623212926 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:56.316683 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 687/1367] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:47:56.317212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-95f74dd8-ad8a-4b5d-9de3-18b623212926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:56.321388 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b645d-d2b3-41d3-a40f-12c8b62b2734 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:56.326205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3e6b645d-d2b3-41d3-a40f-12c8b62b2734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:56.351764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b645d-d2b3-41d3-a40f-12c8b62b2734 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:56.367773 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfcac1-2b40-4de8-b88d-cf11ec40cc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:56.368732 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfcac1-2b40-4de8-b88d-cf11ec40cc1c admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:56.369105 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfcac1-2b40-4de8-b88d-cf11ec40cc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:56.369546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfcac1-2b40-4de8-b88d-cf11ec40cc1c admin admin] PATH_INFO: `/v3/projects/6cfd12ceb2a5454281d74296ac59f5c7/users/df70d29f9b35498c8e4704b59f98efbf/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:56.374241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfcac1-2b40-4de8-b88d-cf11ec40cc1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfd12ceb2a5454281d74296ac59f5c7, user_id=df70d29f9b35498c8e4704b59f98efbf, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:56.379863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfcac1-2b40-4de8-b88d-cf11ec40cc1c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:56.405832 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 688/1368] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:47:56 2022] PUT /identity/v3/projects/6cfd12ceb2a5454281d74296ac59f5c7/users/df70d29f9b35498c8e4704b59f98efbf/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:47:56.416894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669c17d-7fe8-4e6d-831c-d5ddeaed7531 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:56.417329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669c17d-7fe8-4e6d-831c-d5ddeaed7531 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:56.417692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669c17d-7fe8-4e6d-831c-d5ddeaed7531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:56.438821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f74dd8-ad8a-4b5d-9de3-18b623212926 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:56.492756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4669c17d-7fe8-4e6d-831c-d5ddeaed7531 None None] MFA Rules not processed for user `df70d29f9b35498c8e4704b59f98efbf`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:56.494076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4669c17d-7fe8-4e6d-831c-d5ddeaed7531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:56.547200 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690c970-0070-4e9d-b1fb-97ff7de8e030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:56.548606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690c970-0070-4e9d-b1fb-97ff7de8e030 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:56.549248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690c970-0070-4e9d-b1fb-97ff7de8e030 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:56.549720 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690c970-0070-4e9d-b1fb-97ff7de8e030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:56.621788 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 689/1369] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:47:56.635951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669c17d-7fe8-4e6d-831c-d5ddeaed7531 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:56.637930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4669c17d-7fe8-4e6d-831c-d5ddeaed7531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:56.670100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690c970-0070-4e9d-b1fb-97ff7de8e030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:56.673478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690c970-0070-4e9d-b1fb-97ff7de8e030 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:56.699218 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 681/1370] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:47:56.730832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031a9ec-87a1-4a03-99d0-d51962e60649 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:56.731379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031a9ec-87a1-4a03-99d0-d51962e60649 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:56.731763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031a9ec-87a1-4a03-99d0-d51962e60649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:56.763436 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669c17d-7fe8-4e6d-831c-d5ddeaed7531 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:56.773756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c14e-769b-4399-919a-34f05cf92152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:56.775030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c14e-769b-4399-919a-34f05cf92152 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:56.775406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c14e-769b-4399-919a-34f05cf92152 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:56.775794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c14e-769b-4399-919a-34f05cf92152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:56.792084 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b031a9ec-87a1-4a03-99d0-d51962e60649 None None] MFA Rules not processed for user `8695364e58b04d829b12ae1b49757536`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:56.794682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b031a9ec-87a1-4a03-99d0-d51962e60649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:56.797243 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c14e-769b-4399-919a-34f05cf92152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:56.799141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c14e-769b-4399-919a-34f05cf92152 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:56.821174 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 690/1371] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:56.843634 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:56.898241 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 682/1372] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:56.911675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b0e8-aa54-4e4b-8235-740b2d70caad None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:56.912271 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b0e8-aa54-4e4b-8235-740b2d70caad None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:56.912770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b0e8-aa54-4e4b-8235-740b2d70caad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:56.958406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031a9ec-87a1-4a03-99d0-d51962e60649 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:56.975654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031a9ec-87a1-4a03-99d0-d51962e60649 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:56.983435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd13e3-30a4-4f71-9c33-77913e8a4c2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:56.984758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd13e3-30a4-4f71-9c33-77913e8a4c2a service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:56.985500 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f7e3b0e8-aa54-4e4b-8235-740b2d70caad None None] MFA Rules not processed for user `8695364e58b04d829b12ae1b49757536`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:56.985889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd13e3-30a4-4f71-9c33-77913e8a4c2a service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:56.986774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd13e3-30a4-4f71-9c33-77913e8a4c2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:56.988809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f7e3b0e8-aa54-4e4b-8235-740b2d70caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:56.990993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6ddd13e3-30a4-4f71-9c33-77913e8a4c2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:57.006898 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 691/1373] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:57.021747 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b0e8-aa54-4e4b-8235-740b2d70caad None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:57.026069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f7e3b0e8-aa54-4e4b-8235-740b2d70caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:57.040626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:57.200308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd13e3-30a4-4f71-9c33-77913e8a4c2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:57.201803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd13e3-30a4-4f71-9c33-77913e8a4c2a service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:57.219586 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 683/1374] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:47:57.288103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b48f-0491-40e5-bef7-a7f110fb3d70 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:57.288404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b48f-0491-40e5-bef7-a7f110fb3d70 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:57.288684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b48f-0491-40e5-bef7-a7f110fb3d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:57.309755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b0e8-aa54-4e4b-8235-740b2d70caad None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:57.328948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a8f1b-c3d8-48d1-a615-68585f7d992b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:57.329614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a8f1b-c3d8-48d1-a615-68585f7d992b service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:57.329892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a8f1b-c3d8-48d1-a615-68585f7d992b service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:57.330160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a8f1b-c3d8-48d1-a615-68585f7d992b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:57.352090 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a8f1b-c3d8-48d1-a615-68585f7d992b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:57.354426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a8f1b-c3d8-48d1-a615-68585f7d992b service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:57.373476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-acb6b48f-0491-40e5-bef7-a7f110fb3d70 None None] MFA Rules not processed for user `43af1b9b66804624aff6eef2f435d581`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:57.376390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-acb6b48f-0491-40e5-bef7-a7f110fb3d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:57.382764 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 692/1375] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:57.781376 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 684/1376] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:47:57.852357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c611243-9ad6-44a5-9506-7a100d18cd01 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:57.852649 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c611243-9ad6-44a5-9506-7a100d18cd01 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:57.852933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c611243-9ad6-44a5-9506-7a100d18cd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:57.928880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8c611243-9ad6-44a5-9506-7a100d18cd01 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:57.930115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8c611243-9ad6-44a5-9506-7a100d18cd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:58.014331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6b48f-0491-40e5-bef7-a7f110fb3d70 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:58.015814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-acb6b48f-0491-40e5-bef7-a7f110fb3d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:58.055279 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 693/1377] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:58.074403 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6b48f-0491-40e5-bef7-a7f110fb3d70 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:58.090188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f57b8-5de2-4b7e-a08d-2fb251738a9a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:58.090188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f57b8-5de2-4b7e-a08d-2fb251738a9a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:58.090188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f57b8-5de2-4b7e-a08d-2fb251738a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:58.094012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a9b8c-a576-4760-ac05-39f9b3ca8643 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:58.111451 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a9b8c-a576-4760-ac05-39f9b3ca8643 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:58.116377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a9b8c-a576-4760-ac05-39f9b3ca8643 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:58.116679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a9b8c-a576-4760-ac05-39f9b3ca8643 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:58.127367 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-166f57b8-5de2-4b7e-a08d-2fb251738a9a None None] MFA Rules not processed for user `712e5cf4cc004918817ab1802b9c69d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:58.130128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-166f57b8-5de2-4b7e-a08d-2fb251738a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:58.137905 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a9b8c-a576-4760-ac05-39f9b3ca8643 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:58.142998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a9b8c-a576-4760-ac05-39f9b3ca8643 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:58.159575 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 685/1378] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:47:58.161407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea430-3b74-4e7c-9a73-3635996a338e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:58.161807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea430-3b74-4e7c-9a73-3635996a338e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:58.161969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea430-3b74-4e7c-9a73-3635996a338e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:58.235305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6ceea430-3b74-4e7c-9a73-3635996a338e None None] MFA Rules not processed for user `2b6b4f711c354d279cdb164c093d6a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:58.236671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6ceea430-3b74-4e7c-9a73-3635996a338e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:58.287315 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 694/1379] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:58.345536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:58.412912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdc570-35e0-463f-b083-7cc65d47cab1 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:58.413561 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdc570-35e0-463f-b083-7cc65d47cab1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:58.413915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdc570-35e0-463f-b083-7cc65d47cab1 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:58.415869 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 695/1380] 10.222.0.26 () {54 vars in 912 bytes} [Mon Oct 24 08:47:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:47:58.421194 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 686/1381] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:47:58.441911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb4892-131f-4a7d-8bda-d8dbd735b4ea None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:58.442306 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb4892-131f-4a7d-8bda-d8dbd735b4ea None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:58.442811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb4892-131f-4a7d-8bda-d8dbd735b4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:58.460310 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceea430-3b74-4e7c-9a73-3635996a338e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:58.463002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6ceea430-3b74-4e7c-9a73-3635996a338e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:58.464179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:58.549801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e0cb4892-131f-4a7d-8bda-d8dbd735b4ea None None] MFA Rules not processed for user `2254623f523c48f0aa454b85be41df95`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:47:58.551230 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e0cb4892-131f-4a7d-8bda-d8dbd735b4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:58.566794 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 696/1382] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:47:58.597440 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb4892-131f-4a7d-8bda-d8dbd735b4ea None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:58.598908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e0cb4892-131f-4a7d-8bda-d8dbd735b4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:58.626041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb4892-131f-4a7d-8bda-d8dbd735b4ea None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:58.640569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:58.641236 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:58.641513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:58.641906 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] PATH_INFO: `/v3/projects/ca706bc9087142fdb0941f08e782d7d0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:58.643160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca706bc9087142fdb0941f08e782d7d0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:58.645693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:58.773187 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceea430-3b74-4e7c-9a73-3635996a338e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:58.808853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca706bc9087142fdb0941f08e782d7d0', 'request_id': 'req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:58.832854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca706bc9087142fdb0941f08e782d7d0', 'request_id': 'req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:58.858610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca706bc9087142fdb0941f08e782d7d0', 'request_id': 'req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:58.895257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] The token cache is being invalidate because project ca706bc9087142fdb0941f08e782d7d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:47:58.895601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:58.896902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:58.910429 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 697/1383] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:47:58 2022] DELETE /identity/v3/projects/ca706bc9087142fdb0941f08e782d7d0 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:47:58.911588 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:58.924893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:59.031305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0f6a8-9bf2-4aed-9557-96d8a08dea08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:59.031893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0f6a8-9bf2-4aed-9557-96d8a08dea08 service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:59.031893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0f6a8-9bf2-4aed-9557-96d8a08dea08 service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:59.032083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0f6a8-9bf2-4aed-9557-96d8a08dea08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:59.036194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-46e0f6a8-9bf2-4aed-9557-96d8a08dea08 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:59.116643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388e4a-21ae-4348-8f7e-5dee5aa2cb7c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:59.132003 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:59.133800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:59.133976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:59.139222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:59.145799 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:59.187903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:59.189406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:59.214088 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 698/1384] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:47:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:47:59.214962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:47:59.216646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:59.217709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:47:59.226365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0f6a8-9bf2-4aed-9557-96d8a08dea08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:59.229205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e0f6a8-9bf2-4aed-9557-96d8a08dea08 service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:59.256465 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 687/1385] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:47:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 797 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:47:59.498104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecd7c4-f213-4720-8bfe-4132dff5b47b service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:47:59.506666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:47:59.507326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:47:59.508816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:47:59.510143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] PATH_INFO: `/v3/users/9cb17b317a1d4aeb82570db8acb0fa61` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:47:59.512802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb17b317a1d4aeb82570db8acb0fa61)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:47:59.519051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:47:59.598801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:47:59.675288 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb17b317a1d4aeb82570db8acb0fa61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:59.703084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb17b317a1d4aeb82570db8acb0fa61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:59.704847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb17b317a1d4aeb82570db8acb0fa61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:59.720331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb17b317a1d4aeb82570db8acb0fa61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:59.755738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb17b317a1d4aeb82570db8acb0fa61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:47:59.770113 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 699/1386] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:47:59 2022] DELETE /identity/v3/users/9cb17b317a1d4aeb82570db8acb0fa61 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:01.513807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b3c24-962e-43fc-8515-9865f729bed7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:01.514625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b3c24-962e-43fc-8515-9865f729bed7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:01.514625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b3c24-962e-43fc-8515-9865f729bed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:01.654479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a03b3c24-962e-43fc-8515-9865f729bed7 None None] MFA Rules not processed for user `2254623f523c48f0aa454b85be41df95`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:01.662004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a03b3c24-962e-43fc-8515-9865f729bed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:01.991861 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 688/1387] 10.222.0.36 () {58 vars in 1015 bytes} [Mon Oct 24 08:48:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:02.149188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:02.151501 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:02.184169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3dadc-b519-4544-88b3-f0992636167f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:02.208999 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896bdf7-53f8-45ee-9929-2cf39607abde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:02.209877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896bdf7-53f8-45ee-9929-2cf39607abde service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:02.213455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896bdf7-53f8-45ee-9929-2cf39607abde service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:02.213876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896bdf7-53f8-45ee-9929-2cf39607abde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:02.264792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896bdf7-53f8-45ee-9929-2cf39607abde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:02.277814 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896bdf7-53f8-45ee-9929-2cf39607abde service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:02.311163 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 700/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:03.428704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9d08-6490-4312-9ccb-9aab643d1521 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:03.429605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9d08-6490-4312-9ccb-9aab643d1521 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:03.430052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9d08-6490-4312-9ccb-9aab643d1521 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:03.431940 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 689/1389] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:48:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:48:03.455947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75020db8-1b0a-4380-9976-2070cff37b98 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:03.456594 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75020db8-1b0a-4380-9976-2070cff37b98 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:03.456791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75020db8-1b0a-4380-9976-2070cff37b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:03.520820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-75020db8-1b0a-4380-9976-2070cff37b98 None None] MFA Rules not processed for user `c3749568726644be9ebdf4d87463ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:03.532374 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-75020db8-1b0a-4380-9976-2070cff37b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:03.657308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9d08-6490-4312-9ccb-9aab643d1521 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:03.659607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-28ed9d08-6490-4312-9ccb-9aab643d1521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:03.810115 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 701/1390] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:48:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:03.853536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9d08-6490-4312-9ccb-9aab643d1521 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:03.864390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:03.865427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:03.865919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:03.867353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] PATH_INFO: `/v3/projects/4f5cd4ccdd0d4f81b280c1d9ed96e359` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:03.876838 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f5cd4ccdd0d4f81b280c1d9ed96e359)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:03.879926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:03.963777 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada062b4-0463-4da5-9b5e-7952164c7f9d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:03.964204 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada062b4-0463-4da5-9b5e-7952164c7f9d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:03.964562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada062b4-0463-4da5-9b5e-7952164c7f9d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:03.966526 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 702/1391] 10.222.0.26 () {54 vars in 961 bytes} [Mon Oct 24 08:48:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:48:03.972190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f93165-c2b2-4b1a-8242-e860a4d5843b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:03.972465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f93165-c2b2-4b1a-8242-e860a4d5843b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:03.972837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f93165-c2b2-4b1a-8242-e860a4d5843b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:04.006105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f5cd4ccdd0d4f81b280c1d9ed96e359', 'request_id': 'req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:04.030051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f5cd4ccdd0d4f81b280c1d9ed96e359', 'request_id': 'req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:04.052221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f5cd4ccdd0d4f81b280c1d9ed96e359', 'request_id': 'req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:04.053147 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a9f93165-c2b2-4b1a-8242-e860a4d5843b None None] MFA Rules not processed for user `c3749568726644be9ebdf4d87463ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:04.059595 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a9f93165-c2b2-4b1a-8242-e860a4d5843b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:04.123033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] The token cache is being invalidate because project 4f5cd4ccdd0d4f81b280c1d9ed96e359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:04.123506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:04.126623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66768a94-4ffc-4de1-becf-1fbf7aa83ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:04.129608 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 690/1392] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:03 2022] DELETE /identity/v3/projects/4f5cd4ccdd0d4f81b280c1d9ed96e359 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:04.135058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790df0a1-dec5-40a3-aba6-c40d93eaf18e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:04.136481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790df0a1-dec5-40a3-aba6-c40d93eaf18e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:04.136949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790df0a1-dec5-40a3-aba6-c40d93eaf18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:04.199583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-790df0a1-dec5-40a3-aba6-c40d93eaf18e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:04.200866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-790df0a1-dec5-40a3-aba6-c40d93eaf18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:04.240919 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 703/1393] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Oct 24 08:48:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:04.251392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f93165-c2b2-4b1a-8242-e860a4d5843b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:04.269653 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f93165-c2b2-4b1a-8242-e860a4d5843b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:04.293285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9665772-9cad-417f-b254-9e52f4696ac8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:04.294183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9665772-9cad-417f-b254-9e52f4696ac8 service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:04.294770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9665772-9cad-417f-b254-9e52f4696ac8 service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:04.295292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9665772-9cad-417f-b254-9e52f4696ac8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:04.309089 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f9665772-9cad-417f-b254-9e52f4696ac8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:04.313552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:04.347339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9665772-9cad-417f-b254-9e52f4696ac8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:04.350187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9665772-9cad-417f-b254-9e52f4696ac8 service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:04.401698 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 704/1394] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:48:04.421543 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 691/1395] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:04.443693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687367fa-5a65-48cc-8f14-7ba7f6cf4c7c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:04.444353 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687367fa-5a65-48cc-8f14-7ba7f6cf4c7c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:04.446042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687367fa-5a65-48cc-8f14-7ba7f6cf4c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:04.548568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-687367fa-5a65-48cc-8f14-7ba7f6cf4c7c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:04.550310 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-687367fa-5a65-48cc-8f14-7ba7f6cf4c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:04.575185 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 705/1396] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:04.579754 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790df0a1-dec5-40a3-aba6-c40d93eaf18e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:04.606418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790df0a1-dec5-40a3-aba6-c40d93eaf18e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:04.635880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cb577-1aa0-4399-a710-e93e18b528fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:04.636708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb577-1aa0-4399-a710-e93e18b528fa admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:04.637208 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb577-1aa0-4399-a710-e93e18b528fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:04.637681 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cb577-1aa0-4399-a710-e93e18b528fa admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:04.638574 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cb577-1aa0-4399-a710-e93e18b528fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:04.639104 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cb577-1aa0-4399-a710-e93e18b528fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:04.640975 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cb577-1aa0-4399-a710-e93e18b528fa admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:04.676770 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 692/1397] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:04.678266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687367fa-5a65-48cc-8f14-7ba7f6cf4c7c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:04.686841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:04.694186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687367fa-5a65-48cc-8f14-7ba7f6cf4c7c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:04.705976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737e6ce-5cfc-4326-9bf4-6f2841c3a9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:04.707085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737e6ce-5cfc-4326-9bf4-6f2841c3a9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:04.707462 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737e6ce-5cfc-4326-9bf4-6f2841c3a9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:04.707830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737e6ce-5cfc-4326-9bf4-6f2841c3a9a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:04.708726 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737e6ce-5cfc-4326-9bf4-6f2841c3a9a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:04.711078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737e6ce-5cfc-4326-9bf4-6f2841c3a9a5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:04.772503 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 706/1398] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:04 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:48:04.776825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cb577-1aa0-4399-a710-e93e18b528fa admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:04.791458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cb577-1aa0-4399-a710-e93e18b528fa admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:04.934398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3e417-4450-4eea-b568-8bee9b765bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:04.935438 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3e417-4450-4eea-b568-8bee9b765bef admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:04.935927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3e417-4450-4eea-b568-8bee9b765bef admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:04.936395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3e417-4450-4eea-b568-8bee9b765bef admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:04.937355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3e417-4450-4eea-b568-8bee9b765bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:04.939542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3e417-4450-4eea-b568-8bee9b765bef admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:04.969376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861aebc-6946-478d-b08b-11367f9bea42 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:04.969669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861aebc-6946-478d-b08b-11367f9bea42 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:04.969949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861aebc-6946-478d-b08b-11367f9bea42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:04.998677 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 693/1399] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:04 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:04.999907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3e417-4450-4eea-b568-8bee9b765bef admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:05.022978 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3e417-4450-4eea-b568-8bee9b765bef admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:05.039296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5963808-8f5b-4cbc-a7b8-2a7108b8b6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:05.040607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5963808-8f5b-4cbc-a7b8-2a7108b8b6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:05.041103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5963808-8f5b-4cbc-a7b8-2a7108b8b6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:05.041850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5963808-8f5b-4cbc-a7b8-2a7108b8b6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:05.042701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5963808-8f5b-4cbc-a7b8-2a7108b8b6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:05.048442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5963808-8f5b-4cbc-a7b8-2a7108b8b6a0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:05.072351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c861aebc-6946-478d-b08b-11367f9bea42 None None] MFA Rules not processed for user `8695364e58b04d829b12ae1b49757536`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:05.079788 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c861aebc-6946-478d-b08b-11367f9bea42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:05.083116 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 694/1400] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:05.084701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5963808-8f5b-4cbc-a7b8-2a7108b8b6a0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:05.108517 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5963808-8f5b-4cbc-a7b8-2a7108b8b6a0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:05.128657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdaf16-6f44-4628-980e-27b519526341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:05.129675 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdaf16-6f44-4628-980e-27b519526341 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:05.130103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdaf16-6f44-4628-980e-27b519526341 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:05.130524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdaf16-6f44-4628-980e-27b519526341 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:05.131319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdaf16-6f44-4628-980e-27b519526341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:05.133841 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bdaf16-6f44-4628-980e-27b519526341 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:05.154749 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 695/1401] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:05.161160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdaf16-6f44-4628-980e-27b519526341 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:05.189395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdaf16-6f44-4628-980e-27b519526341 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:05.201814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77440a18-01f3-4f39-b369-39538034fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:05.202599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77440a18-01f3-4f39-b369-39538034fc9a admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:05.202983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77440a18-01f3-4f39-b369-39538034fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:05.203358 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77440a18-01f3-4f39-b369-39538034fc9a admin admin] PATH_INFO: `/v3/projects/589845ed3e4343ae819a82e54e5de657/users/fa86153377034e08983e2accabbabe7d/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:05.242265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77440a18-01f3-4f39-b369-39538034fc9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=589845ed3e4343ae819a82e54e5de657, user_id=fa86153377034e08983e2accabbabe7d, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:05.245127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77440a18-01f3-4f39-b369-39538034fc9a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:05.285885 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 707/1402] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:05.310773 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 696/1403] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:05 2022] PUT /identity/v3/projects/589845ed3e4343ae819a82e54e5de657/users/fa86153377034e08983e2accabbabe7d/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:05.312696 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a022-ad7e-4df9-9d7b-121698de9795 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:05.313675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a022-ad7e-4df9-9d7b-121698de9795 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:05.314297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a022-ad7e-4df9-9d7b-121698de9795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:05.409945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-60b1a022-ad7e-4df9-9d7b-121698de9795 None None] MFA Rules not processed for user `fa86153377034e08983e2accabbabe7d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:05.411344 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-60b1a022-ad7e-4df9-9d7b-121698de9795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:05.748886 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fb5a9-e3ef-4dde-a5ce-607985eeb145 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:05.749913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fb5a9-e3ef-4dde-a5ce-607985eeb145 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:05.750345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fb5a9-e3ef-4dde-a5ce-607985eeb145 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:05.753722 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 697/1404] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:48:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:48:05.755561 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 708/1405] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:05.759695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fb5a9-e3ef-4dde-a5ce-607985eeb145 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:06.012052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fb5a9-e3ef-4dde-a5ce-607985eeb145 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:06.044647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533e483-2def-45ec-9a11-d546b6f85d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:06.045433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e483-2def-45ec-9a11-d546b6f85d49 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:06.045711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e483-2def-45ec-9a11-d546b6f85d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:06.046004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e483-2def-45ec-9a11-d546b6f85d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:06.046624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533e483-2def-45ec-9a11-d546b6f85d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:06.051225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533e483-2def-45ec-9a11-d546b6f85d49 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:06.092448 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:06.136213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add0fc8-49f5-4f55-9287-6b342c60d402 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:06.136997 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add0fc8-49f5-4f55-9287-6b342c60d402 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:06.137472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add0fc8-49f5-4f55-9287-6b342c60d402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:06.170643 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 698/1406] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:05 2022] POST /identity/v3/projects => generated 389 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:06.174777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533e483-2def-45ec-9a11-d546b6f85d49 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:06.202516 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4add0fc8-49f5-4f55-9287-6b342c60d402 None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:06.204965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4add0fc8-49f5-4f55-9287-6b342c60d402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:06.458691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533e483-2def-45ec-9a11-d546b6f85d49 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:06.479705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03db8c-f8f4-4144-87ba-8c5081229d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:06.480700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03db8c-f8f4-4144-87ba-8c5081229d01 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:06.480700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03db8c-f8f4-4144-87ba-8c5081229d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:06.480946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03db8c-f8f4-4144-87ba-8c5081229d01 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:06.481601 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03db8c-f8f4-4144-87ba-8c5081229d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:06.488093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03db8c-f8f4-4144-87ba-8c5081229d01 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:06.518449 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 709/1407] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:48:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:06.558640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add0fc8-49f5-4f55-9287-6b342c60d402 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:06.559574 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 699/1408] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:06 2022] POST /identity/v3/users => generated 301 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:06.573514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add0fc8-49f5-4f55-9287-6b342c60d402 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:06.588042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec98aae-e6ed-46c0-8f23-6a02cd621a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:06.588903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec98aae-e6ed-46c0-8f23-6a02cd621a1a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:06.589336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec98aae-e6ed-46c0-8f23-6a02cd621a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:06.589760 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec98aae-e6ed-46c0-8f23-6a02cd621a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:06.590582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec98aae-e6ed-46c0-8f23-6a02cd621a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:06.592460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec98aae-e6ed-46c0-8f23-6a02cd621a1a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:06.599406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:06.622351 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 710/1409] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:48:06.627300 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03db8c-f8f4-4144-87ba-8c5081229d01 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:06.643932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03db8c-f8f4-4144-87ba-8c5081229d01 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:06.657427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb0a0-2449-4ab8-94a9-0a0ec80f0e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:06.658473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb0a0-2449-4ab8-94a9-0a0ec80f0e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:06.659765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb0a0-2449-4ab8-94a9-0a0ec80f0e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:06.660219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb0a0-2449-4ab8-94a9-0a0ec80f0e1a admin admin] PATH_INFO: `/v3/projects/4b319d17a0114b038d3a8fa0985f727a/users/c65566c151eb4dc9a65243d1b25deeec/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:06.710265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adb0a0-2449-4ab8-94a9-0a0ec80f0e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b319d17a0114b038d3a8fa0985f727a, user_id=c65566c151eb4dc9a65243d1b25deeec, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:06.713362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adb0a0-2449-4ab8-94a9-0a0ec80f0e1a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:06.740124 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 700/1410] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:06 2022] PUT /identity/v3/projects/4b319d17a0114b038d3a8fa0985f727a/users/c65566c151eb4dc9a65243d1b25deeec/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:06.744282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec98aae-e6ed-46c0-8f23-6a02cd621a1a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:06.759464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec98aae-e6ed-46c0-8f23-6a02cd621a1a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:06.909918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9489b1-075d-4f06-b905-74829902558a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:06.911527 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9489b1-075d-4f06-b905-74829902558a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:06.912898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9489b1-075d-4f06-b905-74829902558a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:06.913322 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9489b1-075d-4f06-b905-74829902558a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:06.916422 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9489b1-075d-4f06-b905-74829902558a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:06.921857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9489b1-075d-4f06-b905-74829902558a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:06.951200 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 711/1411] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:06.954139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb0a0-2449-4ab8-94a9-0a0ec80f0e1a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:06.973728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb0a0-2449-4ab8-94a9-0a0ec80f0e1a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:06.987747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c302736-72a5-434a-a614-ad95cd6824e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:06.988482 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c302736-72a5-434a-a614-ad95cd6824e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:06.991470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c302736-72a5-434a-a614-ad95cd6824e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:06.991746 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c302736-72a5-434a-a614-ad95cd6824e7 admin admin] PATH_INFO: `/v3/projects/4b319d17a0114b038d3a8fa0985f727a/users/c65566c151eb4dc9a65243d1b25deeec/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:07.001649 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c302736-72a5-434a-a614-ad95cd6824e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b319d17a0114b038d3a8fa0985f727a, user_id=c65566c151eb4dc9a65243d1b25deeec, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:07.011113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c302736-72a5-434a-a614-ad95cd6824e7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:07.054779 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 701/1412] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:06 2022] PUT /identity/v3/projects/4b319d17a0114b038d3a8fa0985f727a/users/c65566c151eb4dc9a65243d1b25deeec/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:07.065230 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94659b-89fb-4afb-bca0-67471dd82c00 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:07.065763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94659b-89fb-4afb-bca0-67471dd82c00 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:07.065883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94659b-89fb-4afb-bca0-67471dd82c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:07.139474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1b94659b-89fb-4afb-bca0-67471dd82c00 None None] MFA Rules not processed for user `c65566c151eb4dc9a65243d1b25deeec`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:07.141599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1b94659b-89fb-4afb-bca0-67471dd82c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:07.410827 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 712/1413] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:07.432336 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c302736-72a5-434a-a614-ad95cd6824e7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:07.434622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3c302736-72a5-434a-a614-ad95cd6824e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:07.612535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c302736-72a5-434a-a614-ad95cd6824e7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:07.619510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020d4c7-ba2f-4c68-a2ce-f2a97f9bb3ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:07.620141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020d4c7-ba2f-4c68-a2ce-f2a97f9bb3ea service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:07.620422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020d4c7-ba2f-4c68-a2ce-f2a97f9bb3ea service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:07.620720 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020d4c7-ba2f-4c68-a2ce-f2a97f9bb3ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:07.646940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020d4c7-ba2f-4c68-a2ce-f2a97f9bb3ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:07.648416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020d4c7-ba2f-4c68-a2ce-f2a97f9bb3ea service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:07.673852 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 702/1414] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:07 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:08.017712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94659b-89fb-4afb-bca0-67471dd82c00 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:08.020044 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1b94659b-89fb-4afb-bca0-67471dd82c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:08.116533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020d4c7-ba2f-4c68-a2ce-f2a97f9bb3ea service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:08.161339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94659b-89fb-4afb-bca0-67471dd82c00 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:08.167279 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:08.168510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:08.168859 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:08.169195 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] PATH_INFO: `/v3/users/9fe3ac847852457db7f533067f2c1337` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:08.172466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe3ac847852457db7f533067f2c1337)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:08.186861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:08.199448 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:08.220060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:08.240947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020d4c7-ba2f-4c68-a2ce-f2a97f9bb3ea service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:08.265802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:08.267362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:08.267362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:08.267362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] PATH_INFO: `/v3/users/fa86153377034e08983e2accabbabe7d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:08.268280 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa86153377034e08983e2accabbabe7d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:08.273341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:08.294066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe3ac847852457db7f533067f2c1337'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.317562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe3ac847852457db7f533067f2c1337'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.323197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe3ac847852457db7f533067f2c1337'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.345765 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe3ac847852457db7f533067f2c1337'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.397177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6ca2f2cd-f143-498c-8c2b-ef469c0ad7d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe3ac847852457db7f533067f2c1337'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.421059 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 713/1415] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:08 2022] DELETE /identity/v3/users/9fe3ac847852457db7f533067f2c1337 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:08.426316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df158e9-9ad1-4bc6-9b8d-c1d75ef76765 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:08.426568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df158e9-9ad1-4bc6-9b8d-c1d75ef76765 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:08.426865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df158e9-9ad1-4bc6-9b8d-c1d75ef76765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:08.430784 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa86153377034e08983e2accabbabe7d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.435381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa86153377034e08983e2accabbabe7d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.460332 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa86153377034e08983e2accabbabe7d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.486756 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa86153377034e08983e2accabbabe7d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.503140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7df158e9-9ad1-4bc6-9b8d-c1d75ef76765 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:08.508166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7df158e9-9ad1-4bc6-9b8d-c1d75ef76765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:08.535121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-35a30ee4-7659-4dda-8096-01ffe503fcae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa86153377034e08983e2accabbabe7d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:08.562121 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 703/1416] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:08 2022] DELETE /identity/v3/users/fa86153377034e08983e2accabbabe7d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:08.563774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e1dce-1275-4b18-afd8-ea32a4792d8c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:08.564062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e1dce-1275-4b18-afd8-ea32a4792d8c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:08.564377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e1dce-1275-4b18-afd8-ea32a4792d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:08.641222 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-413e1dce-1275-4b18-afd8-ea32a4792d8c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:08.642994 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-413e1dce-1275-4b18-afd8-ea32a4792d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:08.740285 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 714/1417] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:08.776867 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 704/1418] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:08.816439 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df158e9-9ad1-4bc6-9b8d-c1d75ef76765 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:08.817856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e1dce-1275-4b18-afd8-ea32a4792d8c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:08.820036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-413e1dce-1275-4b18-afd8-ea32a4792d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:08.821308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7df158e9-9ad1-4bc6-9b8d-c1d75ef76765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:09.056155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df158e9-9ad1-4bc6-9b8d-c1d75ef76765 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:09.058214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e1dce-1275-4b18-afd8-ea32a4792d8c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:09.072386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a1af4-5279-42e2-b36f-70ecf120d021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:09.074066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a1af4-5279-42e2-b36f-70ecf120d021 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:09.074066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a1af4-5279-42e2-b36f-70ecf120d021 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:09.074066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a1af4-5279-42e2-b36f-70ecf120d021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:09.075278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5e66f-5c20-4236-97fd-8f7722493e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:09.076383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5e66f-5c20-4236-97fd-8f7722493e02 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:09.076811 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5e66f-5c20-4236-97fd-8f7722493e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:09.078764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5e66f-5c20-4236-97fd-8f7722493e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:09.108205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a1af4-5279-42e2-b36f-70ecf120d021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:09.112217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a1af4-5279-42e2-b36f-70ecf120d021 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:09.113176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5e66f-5c20-4236-97fd-8f7722493e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:09.115821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5e66f-5c20-4236-97fd-8f7722493e02 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:09.126351 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 715/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:09.148717 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 705/1420] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:09.680378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a1af4-5279-42e2-b36f-70ecf120d021 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:09.680378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-530a1af4-5279-42e2-b36f-70ecf120d021 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:09.708675 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f495e-75ff-4623-b144-d35d955be85d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:09.709259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f495e-75ff-4623-b144-d35d955be85d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:09.710044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f495e-75ff-4623-b144-d35d955be85d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:09.712303 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 706/1421] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:48:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:48:09.792272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:09.926683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a1af4-5279-42e2-b36f-70ecf120d021 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:09.955469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314fd3a-9987-4777-ac10-edfcc440941d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:09.958004 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314fd3a-9987-4777-ac10-edfcc440941d service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:09.959443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314fd3a-9987-4777-ac10-edfcc440941d service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:09.960357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314fd3a-9987-4777-ac10-edfcc440941d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:09.973523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a796f-42bf-4270-aac5-28b67fe6d121 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:09.974099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a796f-42bf-4270-aac5-28b67fe6d121 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:09.974477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a796f-42bf-4270-aac5-28b67fe6d121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:09.978444 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7314fd3a-9987-4777-ac10-edfcc440941d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:10.034312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314fd3a-9987-4777-ac10-edfcc440941d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:10.053382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314fd3a-9987-4777-ac10-edfcc440941d service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:10.059278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cc6a796f-42bf-4270-aac5-28b67fe6d121 None None] MFA Rules not processed for user `43af1b9b66804624aff6eef2f435d581`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:10.067953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cc6a796f-42bf-4270-aac5-28b67fe6d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:10.087853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:10.105326 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 716/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:10.110596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb584b-0784-4436-9579-e1039e7b018b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:10.119218 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb584b-0784-4436-9579-e1039e7b018b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:10.120806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb584b-0784-4436-9579-e1039e7b018b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:10.198857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3abb584b-0784-4436-9579-e1039e7b018b None None] MFA Rules not processed for user `c3749568726644be9ebdf4d87463ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:10.201218 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3abb584b-0784-4436-9579-e1039e7b018b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:10.310409 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 707/1423] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:10.313933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a796f-42bf-4270-aac5-28b67fe6d121 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:10.321527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cc6a796f-42bf-4270-aac5-28b67fe6d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:10.366635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a796f-42bf-4270-aac5-28b67fe6d121 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:10.374350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:10.375566 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:10.376094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:10.376972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] PATH_INFO: `/v3/users/9a0c304691e948ee9e8e462dae7ddc2c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:10.378570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0c304691e948ee9e8e462dae7ddc2c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:10.381111 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:10.472593 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 717/1424] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:48:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:10.533603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb584b-0784-4436-9579-e1039e7b018b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:10.535534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3abb584b-0784-4436-9579-e1039e7b018b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:10.579542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb584b-0784-4436-9579-e1039e7b018b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:10.583309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0c304691e948ee9e8e462dae7ddc2c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:10.584258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0c304691e948ee9e8e462dae7ddc2c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:10.601314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6802e-c7b0-4d72-87fe-cb2009848bb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:10.602689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6802e-c7b0-4d72-87fe-cb2009848bb9 service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:10.605097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6802e-c7b0-4d72-87fe-cb2009848bb9 service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:10.605408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6802e-c7b0-4d72-87fe-cb2009848bb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:10.629434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6802e-c7b0-4d72-87fe-cb2009848bb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:10.631589 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6802e-c7b0-4d72-87fe-cb2009848bb9 service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:10.636361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0c304691e948ee9e8e462dae7ddc2c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:10.664341 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 718/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:48:10.677562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0c304691e948ee9e8e462dae7ddc2c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:10.763902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0c304691e948ee9e8e462dae7ddc2c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:10.788536 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 708/1426] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:10 2022] DELETE /identity/v3/users/9a0c304691e948ee9e8e462dae7ddc2c => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:12.460222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6802e-c7b0-4d72-87fe-cb2009848bb9 service placement] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:12.487813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6802e-c7b0-4d72-87fe-cb2009848bb9 service placement] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:12.683328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:12.686254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:12.735888 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:12.736630 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:12.736912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:12.741574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] PATH_INFO: `/v3/projects/f9c650393b994a0ba7b3576021c21ea2` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:12.749105 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c650393b994a0ba7b3576021c21ea2)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:12.761239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:12.814100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1995c-fece-4f7b-8aee-ca80fc65e925 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:12.823629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:12.825233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:12.825534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:12.825823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] PATH_INFO: `/v3/projects/589845ed3e4343ae819a82e54e5de657` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:12.832303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] RBAC: Authorizing `identity:delete_project(project_id=589845ed3e4343ae819a82e54e5de657)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:12.834814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:12.903837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c650393b994a0ba7b3576021c21ea2', 'request_id': 'req-731515d4-b146-4f45-abf5-1169de4addf3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:12.911018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:12.932997 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c650393b994a0ba7b3576021c21ea2', 'request_id': 'req-731515d4-b146-4f45-abf5-1169de4addf3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:12.943362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c650393b994a0ba7b3576021c21ea2', 'request_id': 'req-731515d4-b146-4f45-abf5-1169de4addf3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:12.949215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:13.026600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] The token cache is being invalidate because project f9c650393b994a0ba7b3576021c21ea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:13.028194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.029390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.033760 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 719/1427] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:12 2022] DELETE /identity/v3/projects/f9c650393b994a0ba7b3576021c21ea2 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:13.034138 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '589845ed3e4343ae819a82e54e5de657', 'request_id': 'req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.037673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:13.042167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:13.047773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '589845ed3e4343ae819a82e54e5de657', 'request_id': 'req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.058848 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '589845ed3e4343ae819a82e54e5de657', 'request_id': 'req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.094601 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] The token cache is being invalidate because project 589845ed3e4343ae819a82e54e5de657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:13.095497 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.097837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.102366 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 709/1428] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:12 2022] DELETE /identity/v3/projects/589845ed3e4343ae819a82e54e5de657 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:13.106236 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:13.109647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:13.163736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731515d4-b146-4f45-abf5-1169de4addf3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:13.249539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb50b9-309d-4a5a-9d5d-7d3769fba50f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:13.258374 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:13.259321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:13.259685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:13.260099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] PATH_INFO: `/v3/users/c65566c151eb4dc9a65243d1b25deeec` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:13.262535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65566c151eb4dc9a65243d1b25deeec)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:13.265013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:13.320171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:13.321454 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:13.321902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:13.322272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] PATH_INFO: `/v3/users/2b6b4f711c354d279cdb164c093d6a8b` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:13.326993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6b4f711c354d279cdb164c093d6a8b)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:13.333442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:13.409781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65566c151eb4dc9a65243d1b25deeec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.412651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65566c151eb4dc9a65243d1b25deeec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.449017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65566c151eb4dc9a65243d1b25deeec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.471510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65566c151eb4dc9a65243d1b25deeec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.526279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65566c151eb4dc9a65243d1b25deeec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.538523 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6b4f711c354d279cdb164c093d6a8b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.544071 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 710/1429] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:13 2022] DELETE /identity/v3/users/c65566c151eb4dc9a65243d1b25deeec => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:13.545141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:13.548892 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:13.557001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6b4f711c354d279cdb164c093d6a8b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.557809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6b4f711c354d279cdb164c093d6a8b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.576733 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6b4f711c354d279cdb164c093d6a8b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.613950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6b4f711c354d279cdb164c093d6a8b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.625845 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 720/1430] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:13 2022] DELETE /identity/v3/users/2b6b4f711c354d279cdb164c093d6a8b => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:13.674196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:13.713126 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc1b4d-d827-4013-b9c9-acb592800815 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:13.798473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:13.801303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:13.801812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:13.802755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] PATH_INFO: `/v3/projects/44f6a6b8b0d24bbf88b3c320bdc21801` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:13.806059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f6a6b8b0d24bbf88b3c320bdc21801)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:13.808408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:13.873390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f6a6b8b0d24bbf88b3c320bdc21801', 'request_id': 'req-66477603-957a-4d91-b6ee-73158a618ed1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.881589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f6a6b8b0d24bbf88b3c320bdc21801', 'request_id': 'req-66477603-957a-4d91-b6ee-73158a618ed1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.908801 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f6a6b8b0d24bbf88b3c320bdc21801', 'request_id': 'req-66477603-957a-4d91-b6ee-73158a618ed1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.928400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] The token cache is being invalidate because project 44f6a6b8b0d24bbf88b3c320bdc21801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:13.928855 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.930210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:13.932099 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 711/1431] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:13 2022] DELETE /identity/v3/projects/44f6a6b8b0d24bbf88b3c320bdc21801 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:13.935964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:13.937376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:13.950231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:13.962872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427df4d1-7b8f-43f1-a46b-8fdca59f6e26 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:14.135878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:14.136570 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:14.136853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:14.137124 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] PATH_INFO: `/v3/users/df70d29f9b35498c8e4704b59f98efbf` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:14.140612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df70d29f9b35498c8e4704b59f98efbf)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:14.142232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:14.228724 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df70d29f9b35498c8e4704b59f98efbf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.238817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df70d29f9b35498c8e4704b59f98efbf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.239802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df70d29f9b35498c8e4704b59f98efbf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.251960 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df70d29f9b35498c8e4704b59f98efbf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.288883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e95db947-1262-470f-9ed0-927fa833c16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df70d29f9b35498c8e4704b59f98efbf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.297504 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 721/1432] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:13 2022] DELETE /identity/v3/users/df70d29f9b35498c8e4704b59f98efbf => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:14.489598 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:14.496016 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:14.523576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66477603-957a-4d91-b6ee-73158a618ed1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:14.641431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:14.642436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:14.642734 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:14.643030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] PATH_INFO: `/v3/projects/bdc465febb3d4e86b20f0ebd00795884` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:14.644448 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc465febb3d4e86b20f0ebd00795884)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:14.646314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:14.673822 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:14.740462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc465febb3d4e86b20f0ebd00795884', 'request_id': 'req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.752015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc465febb3d4e86b20f0ebd00795884', 'request_id': 'req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.760294 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc465febb3d4e86b20f0ebd00795884', 'request_id': 'req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.787561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] The token cache is being invalidate because project bdc465febb3d4e86b20f0ebd00795884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:14.787873 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.789695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8c8dbeb9-ab17-4389-8874-78e5c5c9a33e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:14.792130 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 712/1433] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:14 2022] DELETE /identity/v3/projects/bdc465febb3d4e86b20f0ebd00795884 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:14.798885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996c65a-98cf-4f2f-aee5-73ff5012f0de None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:14.800248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996c65a-98cf-4f2f-aee5-73ff5012f0de None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:14.801108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996c65a-98cf-4f2f-aee5-73ff5012f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:14.825357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c996c65a-98cf-4f2f-aee5-73ff5012f0de None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:14.826849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c996c65a-98cf-4f2f-aee5-73ff5012f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:14.908520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d800cb-4034-4791-a798-681d72fea31c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:14.909267 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d800cb-4034-4791-a798-681d72fea31c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:14.909974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d800cb-4034-4791-a798-681d72fea31c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:14.911920 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 713/1434] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 08:48:14.939183 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c14c2-b625-4aa0-9f26-4ecef5d3dd99 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:14.939466 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c14c2-b625-4aa0-9f26-4ecef5d3dd99 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:14.939737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c14c2-b625-4aa0-9f26-4ecef5d3dd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:14.944876 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 722/1435] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:14.953038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591389a-8a8d-4936-8bbb-719f5645f5d4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:14.955956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591389a-8a8d-4936-8bbb-719f5645f5d4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:14.957978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591389a-8a8d-4936-8bbb-719f5645f5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:14.970143 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-744c14c2-b625-4aa0-9f26-4ecef5d3dd99 None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:14.971819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-744c14c2-b625-4aa0-9f26-4ecef5d3dd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.031335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f591389a-8a8d-4936-8bbb-719f5645f5d4 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:15.032963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f591389a-8a8d-4936-8bbb-719f5645f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.049340 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 723/1436] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:15.051249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591389a-8a8d-4936-8bbb-719f5645f5d4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:15.053638 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f591389a-8a8d-4936-8bbb-719f5645f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.085788 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591389a-8a8d-4936-8bbb-719f5645f5d4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:15.100912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:15.101798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.102171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.102539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] PATH_INFO: `/v3/projects/6cfd12ceb2a5454281d74296ac59f5c7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.104370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfd12ceb2a5454281d74296ac59f5c7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:15.110799 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:15.143770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:15.149074 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 714/1437] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:15.149985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c14c2-b625-4aa0-9f26-4ecef5d3dd99 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:15.165217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c14c2-b625-4aa0-9f26-4ecef5d3dd99 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:15.179995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:15.180629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.180906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.181185 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.181843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:15.182173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:15.183795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:15.199018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfd12ceb2a5454281d74296ac59f5c7', 'request_id': 'req-b73f1942-2312-4abd-901d-9909f6a86bad'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.208749 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 715/1438] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:15.209642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:15.215009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.220841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfd12ceb2a5454281d74296ac59f5c7', 'request_id': 'req-b73f1942-2312-4abd-901d-9909f6a86bad'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.236765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5ae14-15cc-49b4-8c04-e9ba82e530be admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:15.244953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfd12ceb2a5454281d74296ac59f5c7', 'request_id': 'req-b73f1942-2312-4abd-901d-9909f6a86bad'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.278138 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] The token cache is being invalidate because project 6cfd12ceb2a5454281d74296ac59f5c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:15.278614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.279244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.295731 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 724/1439] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:15 2022] DELETE /identity/v3/projects/6cfd12ceb2a5454281d74296ac59f5c7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:15.297329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:15.300715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.365409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:15.366366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.366865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.367327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] PATH_INFO: `/v3/projects/4b319d17a0114b038d3a8fa0985f727a` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.368821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b319d17a0114b038d3a8fa0985f727a)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:15.371870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:15.403882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:15.415442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f1942-2312-4abd-901d-9909f6a86bad admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:15.436945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b319d17a0114b038d3a8fa0985f727a', 'request_id': 'req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.443522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b319d17a0114b038d3a8fa0985f727a', 'request_id': 'req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.469473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b319d17a0114b038d3a8fa0985f727a', 'request_id': 'req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.491961 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] The token cache is being invalidate because project 4b319d17a0114b038d3a8fa0985f727a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:15.492291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.493016 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:15.494897 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 716/1440] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:15 2022] DELETE /identity/v3/projects/4b319d17a0114b038d3a8fa0985f727a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:15.495717 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:15.498000 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.554598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37664926-4f33-406d-ab67-63e48d26fe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:15.555579 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37664926-4f33-406d-ab67-63e48d26fe0c admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.555953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37664926-4f33-406d-ab67-63e48d26fe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.556410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37664926-4f33-406d-ab67-63e48d26fe0c admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.558004 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37664926-4f33-406d-ab67-63e48d26fe0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:15.565542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37664926-4f33-406d-ab67-63e48d26fe0c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:15.644316 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 725/1441] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:15 2022] POST /identity/v3/projects => generated 373 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:15.646117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e62d38-41d9-4889-bc24-daddaf107b50 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.646467 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e62d38-41d9-4889-bc24-daddaf107b50 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.646830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e62d38-41d9-4889-bc24-daddaf107b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.674503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-62e62d38-41d9-4889-bc24-daddaf107b50 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:15.676668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-62e62d38-41d9-4889-bc24-daddaf107b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.689359 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8428aca-b8d2-4bfb-bf9f-4a816bb17a93 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:15.726067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:15.803080 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 726/1442] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:15.814952 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021326b-9cc2-42d4-86c8-0e84961f3906 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.815235 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021326b-9cc2-42d4-86c8-0e84961f3906 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.815584 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021326b-9cc2-42d4-86c8-0e84961f3906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.818061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d43f87-c208-4257-9a45-0727483b401f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:15.818767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d43f87-c208-4257-9a45-0727483b401f service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.819241 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d43f87-c208-4257-9a45-0727483b401f service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.819518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d43f87-c208-4257-9a45-0727483b401f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.823099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-46d43f87-c208-4257-9a45-0727483b401f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.837269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d43f87-c208-4257-9a45-0727483b401f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:15.839460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d43f87-c208-4257-9a45-0727483b401f service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:15.843756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4021326b-9cc2-42d4-86c8-0e84961f3906 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:15.845666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4021326b-9cc2-42d4-86c8-0e84961f3906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.856194 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 727/1443] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:15.857257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021326b-9cc2-42d4-86c8-0e84961f3906 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:15.858666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4021326b-9cc2-42d4-86c8-0e84961f3906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.866868 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 717/1444] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:15.870786 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb7deb-4fed-4c17-a630-2681c85f8e58 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.871528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb7deb-4fed-4c17-a630-2681c85f8e58 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.872962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb7deb-4fed-4c17-a630-2681c85f8e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.883093 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021326b-9cc2-42d4-86c8-0e84961f3906 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:15.888073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fcbe9-0500-4719-b77e-62266ba61d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:15.888971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fcbe9-0500-4719-b77e-62266ba61d7a admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.889433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fcbe9-0500-4719-b77e-62266ba61d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.889908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fcbe9-0500-4719-b77e-62266ba61d7a admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.890759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fcbe9-0500-4719-b77e-62266ba61d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:15.893568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fcbe9-0500-4719-b77e-62266ba61d7a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:15.923475 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 728/1445] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:15 2022] POST /identity/v3/users => generated 287 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:15.924317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6ceb7deb-4fed-4c17-a630-2681c85f8e58 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:15.927879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6ceb7deb-4fed-4c17-a630-2681c85f8e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.928127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea1194-2b36-4b0a-aa9c-c509fd2013e3 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.928590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea1194-2b36-4b0a-aa9c-c509fd2013e3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.929095 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea1194-2b36-4b0a-aa9c-c509fd2013e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.947621 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 718/1446] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:15.949205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb7deb-4fed-4c17-a630-2681c85f8e58 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:15.953704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-09ea1194-2b36-4b0a-aa9c-c509fd2013e3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:15.955163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-09ea1194-2b36-4b0a-aa9c-c509fd2013e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:15.968083 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 729/1447] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:15.969311 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea1194-2b36-4b0a-aa9c-c509fd2013e3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:15.973629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb7deb-4fed-4c17-a630-2681c85f8e58 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:15.983161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea1194-2b36-4b0a-aa9c-c509fd2013e3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:15.987377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e744ba6-d112-42a4-822e-b59c069509bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:15.989431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e744ba6-d112-42a4-822e-b59c069509bc admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.989883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e744ba6-d112-42a4-822e-b59c069509bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.990495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebe1e4-7513-4fd4-8490-8580281587a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:15.990962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e744ba6-d112-42a4-822e-b59c069509bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.991335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebe1e4-7513-4fd4-8490-8580281587a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:15.991704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebe1e4-7513-4fd4-8490-8580281587a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:15.994292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebe1e4-7513-4fd4-8490-8580281587a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:15.994292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebe1e4-7513-4fd4-8490-8580281587a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:15.994541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebe1e4-7513-4fd4-8490-8580281587a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:15.995965 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e744ba6-d112-42a4-822e-b59c069509bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:15.996378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebe1e4-7513-4fd4-8490-8580281587a9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.000021 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e744ba6-d112-42a4-822e-b59c069509bc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.015940 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 730/1448] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:16.018831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebe1e4-7513-4fd4-8490-8580281587a9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.028925 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 719/1449] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:16.030221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e744ba6-d112-42a4-822e-b59c069509bc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.043135 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebe1e4-7513-4fd4-8490-8580281587a9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.049166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62847b59-e226-4a96-8ad7-27ac0c7245a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.049908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62847b59-e226-4a96-8ad7-27ac0c7245a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.050191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62847b59-e226-4a96-8ad7-27ac0c7245a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.050621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62847b59-e226-4a96-8ad7-27ac0c7245a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.051450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62847b59-e226-4a96-8ad7-27ac0c7245a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:16.051768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62847b59-e226-4a96-8ad7-27ac0c7245a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.052674 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e744ba6-d112-42a4-822e-b59c069509bc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.053918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62847b59-e226-4a96-8ad7-27ac0c7245a4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.062080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662384f9-e33b-40b5-a67e-34eef267ff2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.063766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662384f9-e33b-40b5-a67e-34eef267ff2b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.064321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662384f9-e33b-40b5-a67e-34eef267ff2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.064794 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662384f9-e33b-40b5-a67e-34eef267ff2b admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.065804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662384f9-e33b-40b5-a67e-34eef267ff2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.069380 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 731/1450] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:16.070225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662384f9-e33b-40b5-a67e-34eef267ff2b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.070341 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62847b59-e226-4a96-8ad7-27ac0c7245a4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.083103 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62847b59-e226-4a96-8ad7-27ac0c7245a4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.089035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e411-5c0e-45d5-a94b-e12d04cb1c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.089782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e411-5c0e-45d5-a94b-e12d04cb1c63 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.090059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e411-5c0e-45d5-a94b-e12d04cb1c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.090337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e411-5c0e-45d5-a94b-e12d04cb1c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.091150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e411-5c0e-45d5-a94b-e12d04cb1c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.093382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e411-5c0e-45d5-a94b-e12d04cb1c63 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.096791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:16.105713 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 732/1451] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:16.106578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e411-5c0e-45d5-a94b-e12d04cb1c63 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.115190 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 720/1452] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:16 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:16.117560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662384f9-e33b-40b5-a67e-34eef267ff2b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.120465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e411-5c0e-45d5-a94b-e12d04cb1c63 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.193155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c62f3c-7f9a-4bef-a45d-131a5b768196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.193920 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62f3c-7f9a-4bef-a45d-131a5b768196 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.194198 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62f3c-7f9a-4bef-a45d-131a5b768196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.194469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62f3c-7f9a-4bef-a45d-131a5b768196 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.195591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c62f3c-7f9a-4bef-a45d-131a5b768196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.199136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c62f3c-7f9a-4bef-a45d-131a5b768196 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.219497 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662384f9-e33b-40b5-a67e-34eef267ff2b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.225638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba7e90-78e2-4760-9c07-7afc5b7ceaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.226420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba7e90-78e2-4760-9c07-7afc5b7ceaed admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.226726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba7e90-78e2-4760-9c07-7afc5b7ceaed admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.227102 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba7e90-78e2-4760-9c07-7afc5b7ceaed admin admin] PATH_INFO: `/v3/projects/9537d272723b4d4290bbcbdcaf5d9235/users/5e4cace7e4f8479492f24b7d2c3ebc57/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.234691 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 733/1453] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:16 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:16.244075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c62f3c-7f9a-4bef-a45d-131a5b768196 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.245109 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba7e90-78e2-4760-9c07-7afc5b7ceaed admin admin] RBAC: Authorizing `identity:create_grant(project_id=9537d272723b4d4290bbcbdcaf5d9235, user_id=5e4cace7e4f8479492f24b7d2c3ebc57, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.248165 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba7e90-78e2-4760-9c07-7afc5b7ceaed admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.277977 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 721/1454] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:16 2022] PUT /identity/v3/projects/9537d272723b4d4290bbcbdcaf5d9235/users/5e4cace7e4f8479492f24b7d2c3ebc57/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:16.279574 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba7e90-78e2-4760-9c07-7afc5b7ceaed admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.329451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:16.347516 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c62f3c-7f9a-4bef-a45d-131a5b768196 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.384699 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba7e90-78e2-4760-9c07-7afc5b7ceaed admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.390926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd37eda-372e-4437-9d1b-974851f6f3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.391569 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd37eda-372e-4437-9d1b-974851f6f3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.391847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd37eda-372e-4437-9d1b-974851f6f3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.392117 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd37eda-372e-4437-9d1b-974851f6f3f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.392931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd37eda-372e-4437-9d1b-974851f6f3f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.394946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd37eda-372e-4437-9d1b-974851f6f3f4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.427244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a69aa0-e715-4935-afa2-39b0f91d5771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.428027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a69aa0-e715-4935-afa2-39b0f91d5771 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.428508 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a69aa0-e715-4935-afa2-39b0f91d5771 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.428756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a69aa0-e715-4935-afa2-39b0f91d5771 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.429402 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a69aa0-e715-4935-afa2-39b0f91d5771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.431532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a69aa0-e715-4935-afa2-39b0f91d5771 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.442252 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 722/1455] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:16 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:16.442252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863877c1-6c27-4a27-a619-5cb3986e406e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.442252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863877c1-6c27-4a27-a619-5cb3986e406e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.442252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863877c1-6c27-4a27-a619-5cb3986e406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.466520 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 734/1456] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:16 2022] POST /identity/v3/users => generated 285 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:16.467697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a69aa0-e715-4935-afa2-39b0f91d5771 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.480674 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a69aa0-e715-4935-afa2-39b0f91d5771 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.485883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-863877c1-6c27-4a27-a619-5cb3986e406e None None] MFA Rules not processed for user `5e4cace7e4f8479492f24b7d2c3ebc57`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:16.486970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33ffe8-4214-49b6-8a5a-67cb5a3b8925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.487164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-863877c1-6c27-4a27-a619-5cb3986e406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:16.488297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33ffe8-4214-49b6-8a5a-67cb5a3b8925 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.489112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33ffe8-4214-49b6-8a5a-67cb5a3b8925 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.489669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33ffe8-4214-49b6-8a5a-67cb5a3b8925 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.490507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33ffe8-4214-49b6-8a5a-67cb5a3b8925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.492322 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33ffe8-4214-49b6-8a5a-67cb5a3b8925 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.506133 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 735/1457] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:16.507322 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33ffe8-4214-49b6-8a5a-67cb5a3b8925 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.520176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33ffe8-4214-49b6-8a5a-67cb5a3b8925 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.527741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ac480-2130-41c2-ac9a-6419208ace51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.528482 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ac480-2130-41c2-ac9a-6419208ace51 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.529348 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ac480-2130-41c2-ac9a-6419208ace51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.529838 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ac480-2130-41c2-ac9a-6419208ace51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.530572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ac480-2130-41c2-ac9a-6419208ace51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.532270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ac480-2130-41c2-ac9a-6419208ace51 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.547284 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 736/1458] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:16.547284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ac480-2130-41c2-ac9a-6419208ace51 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.558162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ac480-2130-41c2-ac9a-6419208ace51 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.564384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc74229-d201-4f15-b690-ba317b84c8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.565104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc74229-d201-4f15-b690-ba317b84c8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.565506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc74229-d201-4f15-b690-ba317b84c8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.565947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc74229-d201-4f15-b690-ba317b84c8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.566896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc74229-d201-4f15-b690-ba317b84c8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.568393 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:16.570451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc74229-d201-4f15-b690-ba317b84c8a8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.581297 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 723/1459] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:16.582879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863877c1-6c27-4a27-a619-5cb3986e406e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.586821 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 737/1460] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:16.586821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc74229-d201-4f15-b690-ba317b84c8a8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.596367 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863877c1-6c27-4a27-a619-5cb3986e406e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.596592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc74229-d201-4f15-b690-ba317b84c8a8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.603062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301af68d-b887-4f02-bc94-d3f3bdf08e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.603690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301af68d-b887-4f02-bc94-d3f3bdf08e61 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.603969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301af68d-b887-4f02-bc94-d3f3bdf08e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.604237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301af68d-b887-4f02-bc94-d3f3bdf08e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.604335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2c227-dcd8-47bc-9587-a7447ab4dc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.605000 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301af68d-b887-4f02-bc94-d3f3bdf08e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.605113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2c227-dcd8-47bc-9587-a7447ab4dc14 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.605291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2c227-dcd8-47bc-9587-a7447ab4dc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.605620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2c227-dcd8-47bc-9587-a7447ab4dc14 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.606338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2c227-dcd8-47bc-9587-a7447ab4dc14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.606765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301af68d-b887-4f02-bc94-d3f3bdf08e61 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.608788 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2c227-dcd8-47bc-9587-a7447ab4dc14 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.620291 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 724/1461] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:16.621479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301af68d-b887-4f02-bc94-d3f3bdf08e61 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.636365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301af68d-b887-4f02-bc94-d3f3bdf08e61 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.642225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd34ed6-4e8e-4971-a1a8-f25f6b3aaa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.642592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd34ed6-4e8e-4971-a1a8-f25f6b3aaa8d admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.643017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd34ed6-4e8e-4971-a1a8-f25f6b3aaa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.643222 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd34ed6-4e8e-4971-a1a8-f25f6b3aaa8d admin admin] PATH_INFO: `/v3/projects/aae17f90a690492b9921c73f7a6e5a76/users/0559c15f55c942f2ab809c1ea3bfd08f/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.658751 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 738/1462] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:16 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:16.660687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2c227-dcd8-47bc-9587-a7447ab4dc14 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.662598 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd34ed6-4e8e-4971-a1a8-f25f6b3aaa8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae17f90a690492b9921c73f7a6e5a76, user_id=0559c15f55c942f2ab809c1ea3bfd08f, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.665987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd34ed6-4e8e-4971-a1a8-f25f6b3aaa8d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.688235 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 725/1463] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:16 2022] PUT /identity/v3/projects/aae17f90a690492b9921c73f7a6e5a76/users/0559c15f55c942f2ab809c1ea3bfd08f/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:16.689370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd34ed6-4e8e-4971-a1a8-f25f6b3aaa8d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.744530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2c227-dcd8-47bc-9587-a7447ab4dc14 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.766457 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd34ed6-4e8e-4971-a1a8-f25f6b3aaa8d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.773581 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3a87b-d9d3-47d0-9e64-3d2dee64138f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.774505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3a87b-d9d3-47d0-9e64-3d2dee64138f admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.775900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3a87b-d9d3-47d0-9e64-3d2dee64138f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.776417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3a87b-d9d3-47d0-9e64-3d2dee64138f admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.777228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3a87b-d9d3-47d0-9e64-3d2dee64138f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.779829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3a87b-d9d3-47d0-9e64-3d2dee64138f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.795316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:16.820186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5589a-fc16-4dc9-8bde-031b8135cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.820186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5589a-fc16-4dc9-8bde-031b8135cd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.820822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5589a-fc16-4dc9-8bde-031b8135cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.820822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5589a-fc16-4dc9-8bde-031b8135cd2f admin admin] PATH_INFO: `/v3/projects/2dcb344dd87642e89b540eac22f9d3c0/users/81deee2e9696413a86cd88af575dffaa/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.835460 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 726/1464] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:16 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:16.837228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a867d5-97ee-4d3e-b7e7-e9da398f93ae None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.837729 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a867d5-97ee-4d3e-b7e7-e9da398f93ae None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.838624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a867d5-97ee-4d3e-b7e7-e9da398f93ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.840602 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5589a-fc16-4dc9-8bde-031b8135cd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dcb344dd87642e89b540eac22f9d3c0, user_id=81deee2e9696413a86cd88af575dffaa, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.843660 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5589a-fc16-4dc9-8bde-031b8135cd2f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.865387 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 739/1465] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:16 2022] PUT /identity/v3/projects/2dcb344dd87642e89b540eac22f9d3c0/users/81deee2e9696413a86cd88af575dffaa/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:16.866210 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5589a-fc16-4dc9-8bde-031b8135cd2f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:16.879508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-27a867d5-97ee-4d3e-b7e7-e9da398f93ae None None] MFA Rules not processed for user `0559c15f55c942f2ab809c1ea3bfd08f`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:16.880557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-27a867d5-97ee-4d3e-b7e7-e9da398f93ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:16.962995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5589a-fc16-4dc9-8bde-031b8135cd2f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:16.975978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386846b-ff10-497f-8989-9bbd3f0fd7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:16.977019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386846b-ff10-497f-8989-9bbd3f0fd7dd admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.977324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386846b-ff10-497f-8989-9bbd3f0fd7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.977585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386846b-ff10-497f-8989-9bbd3f0fd7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.978356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386846b-ff10-497f-8989-9bbd3f0fd7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:16.980217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386846b-ff10-497f-8989-9bbd3f0fd7dd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:16.986882 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 727/1466] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:16.986882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5443fa-f396-4527-954a-4615f07a3141 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:16.986882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5443fa-f396-4527-954a-4615f07a3141 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:16.986882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5443fa-f396-4527-954a-4615f07a3141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:16.992212 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 740/1467] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:16.997382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386846b-ff10-497f-8989-9bbd3f0fd7dd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.019763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386846b-ff10-497f-8989-9bbd3f0fd7dd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.022733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5c5443fa-f396-4527-954a-4615f07a3141 None None] MFA Rules not processed for user `81deee2e9696413a86cd88af575dffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:17.024205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5c5443fa-f396-4527-954a-4615f07a3141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:17.038925 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84861c63-a6c9-40ad-9cfd-e1078d28457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.039617 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84861c63-a6c9-40ad-9cfd-e1078d28457a admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.039906 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84861c63-a6c9-40ad-9cfd-e1078d28457a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.040484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84861c63-a6c9-40ad-9cfd-e1078d28457a admin admin] PATH_INFO: `/v3/projects/2e2e281a16024323af5bc4dde1405539/users/e71c7541273442a0b3cb2e9bde43bd00/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.064391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84861c63-a6c9-40ad-9cfd-e1078d28457a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2e281a16024323af5bc4dde1405539, user_id=e71c7541273442a0b3cb2e9bde43bd00, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.067248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84861c63-a6c9-40ad-9cfd-e1078d28457a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.085407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:17.087810 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 741/1468] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:16 2022] PUT /identity/v3/projects/2e2e281a16024323af5bc4dde1405539/users/e71c7541273442a0b3cb2e9bde43bd00/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:17.088731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84861c63-a6c9-40ad-9cfd-e1078d28457a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.090488 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-84861c63-a6c9-40ad-9cfd-e1078d28457a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:17.126196 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 728/1469] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:17.127083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5443fa-f396-4527-954a-4615f07a3141 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.137547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5443fa-f396-4527-954a-4615f07a3141 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.144433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84861c63-a6c9-40ad-9cfd-e1078d28457a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.148079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34029d65-e66a-4b9b-8322-be98545ecf1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.148720 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34029d65-e66a-4b9b-8322-be98545ecf1c service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.148995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34029d65-e66a-4b9b-8322-be98545ecf1c service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.149239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34029d65-e66a-4b9b-8322-be98545ecf1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.198328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:17.198627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11b9fe-01ea-4c17-9281-eb6a477237a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.199320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11b9fe-01ea-4c17-9281-eb6a477237a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.199634 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11b9fe-01ea-4c17-9281-eb6a477237a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.200051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11b9fe-01ea-4c17-9281-eb6a477237a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.200864 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11b9fe-01ea-4c17-9281-eb6a477237a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.202517 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11b9fe-01ea-4c17-9281-eb6a477237a6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.209809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34029d65-e66a-4b9b-8322-be98545ecf1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.211430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34029d65-e66a-4b9b-8322-be98545ecf1c service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.212777 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 729/1470] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:17.213555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11b9fe-01ea-4c17-9281-eb6a477237a6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.221692 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 742/1471] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:17.222487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34029d65-e66a-4b9b-8322-be98545ecf1c service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.222734 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11b9fe-01ea-4c17-9281-eb6a477237a6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.227363 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef936ad-61fe-4339-8069-55e2ff50578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.227948 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef936ad-61fe-4339-8069-55e2ff50578a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.228226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef936ad-61fe-4339-8069-55e2ff50578a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.228499 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef936ad-61fe-4339-8069-55e2ff50578a admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.229201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef936ad-61fe-4339-8069-55e2ff50578a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.240777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef936ad-61fe-4339-8069-55e2ff50578a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.244711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34029d65-e66a-4b9b-8322-be98545ecf1c service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.249927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd879cad-dd75-4d3c-9043-7b5c44d84c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.250607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd879cad-dd75-4d3c-9043-7b5c44d84c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.250929 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd879cad-dd75-4d3c-9043-7b5c44d84c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.251281 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd879cad-dd75-4d3c-9043-7b5c44d84c9d admin admin] PATH_INFO: `/v3/projects/2e2e281a16024323af5bc4dde1405539/users/e71c7541273442a0b3cb2e9bde43bd00/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.253348 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd879cad-dd75-4d3c-9043-7b5c44d84c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2e281a16024323af5bc4dde1405539, user_id=e71c7541273442a0b3cb2e9bde43bd00, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.256139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd879cad-dd75-4d3c-9043-7b5c44d84c9d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.264947 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 730/1472] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:17 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:17.266011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef936ad-61fe-4339-8069-55e2ff50578a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.283051 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 743/1473] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:17 2022] PUT /identity/v3/projects/2e2e281a16024323af5bc4dde1405539/users/e71c7541273442a0b3cb2e9bde43bd00/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:17.283051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd879cad-dd75-4d3c-9043-7b5c44d84c9d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.366280 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd879cad-dd75-4d3c-9043-7b5c44d84c9d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.368388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef936ad-61fe-4339-8069-55e2ff50578a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.371663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87a828-c1c6-4065-94ff-4d301425e6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.372390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87a828-c1c6-4065-94ff-4d301425e6af admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.372761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87a828-c1c6-4065-94ff-4d301425e6af admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.373094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87a828-c1c6-4065-94ff-4d301425e6af admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.373680 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87a828-c1c6-4065-94ff-4d301425e6af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.373798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.374313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.374628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.375021 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] PATH_INFO: `/v3/users/0559c15f55c942f2ab809c1ea3bfd08f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.376005 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87a828-c1c6-4065-94ff-4d301425e6af admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.376351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0559c15f55c942f2ab809c1ea3bfd08f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.378447 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.399537 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 744/1474] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:17 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:17.401018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef2229-a33e-4a24-935c-828f0715e9a7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.401385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef2229-a33e-4a24-935c-828f0715e9a7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.401789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef2229-a33e-4a24-935c-828f0715e9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.436110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-acef2229-a33e-4a24-935c-828f0715e9a7 None None] MFA Rules not processed for user `e71c7541273442a0b3cb2e9bde43bd00`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:17.437740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-acef2229-a33e-4a24-935c-828f0715e9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:17.451208 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0559c15f55c942f2ab809c1ea3bfd08f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:17.451890 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0559c15f55c942f2ab809c1ea3bfd08f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:17.453160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:17.463972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0559c15f55c942f2ab809c1ea3bfd08f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:17.481322 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0559c15f55c942f2ab809c1ea3bfd08f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:17.503552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0559c15f55c942f2ab809c1ea3bfd08f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:17.514510 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 731/1475] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:17 2022] DELETE /identity/v3/users/0559c15f55c942f2ab809c1ea3bfd08f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:17.516579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.519098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:17.529168 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 745/1476] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:17.530786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4880ad8-1ea6-44fe-84a3-c29e827b7475 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.531093 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4880ad8-1ea6-44fe-84a3-c29e827b7475 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.531467 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4880ad8-1ea6-44fe-84a3-c29e827b7475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.563551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b4880ad8-1ea6-44fe-84a3-c29e827b7475 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:17.565375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b4880ad8-1ea6-44fe-84a3-c29e827b7475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:17.601907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5045-19b7-49a0-bec9-70c4a66fedbd admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.606790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3ac29-c214-4849-8e3b-ecadbc0a5739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.607396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3ac29-c214-4849-8e3b-ecadbc0a5739 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.607670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3ac29-c214-4849-8e3b-ecadbc0a5739 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.607934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3ac29-c214-4849-8e3b-ecadbc0a5739 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.608570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3ac29-c214-4849-8e3b-ecadbc0a5739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.610144 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3ac29-c214-4849-8e3b-ecadbc0a5739 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.619075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:17.623240 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 732/1477] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:17.623240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3ac29-c214-4849-8e3b-ecadbc0a5739 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.623240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-27e3ac29-c214-4849-8e3b-ecadbc0a5739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:17.634914 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 746/1478] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:17.635724 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4880ad8-1ea6-44fe-84a3-c29e827b7475 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.645207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4880ad8-1ea6-44fe-84a3-c29e827b7475 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.649976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58971b-936c-469f-b679-3bf65286ea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.650631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58971b-936c-469f-b679-3bf65286ea33 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.650940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58971b-936c-469f-b679-3bf65286ea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.651210 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58971b-936c-469f-b679-3bf65286ea33 admin admin] PATH_INFO: `/v3/projects/c6558c9430f548e3bb14d86461d3df79/users/3727a5281e22483bb604ce15c9f6cdfa/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.668593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58971b-936c-469f-b679-3bf65286ea33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6558c9430f548e3bb14d86461d3df79, user_id=3727a5281e22483bb604ce15c9f6cdfa, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.671307 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58971b-936c-469f-b679-3bf65286ea33 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.686982 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 747/1479] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:17 2022] PUT /identity/v3/projects/c6558c9430f548e3bb14d86461d3df79/users/3727a5281e22483bb604ce15c9f6cdfa/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:17.687910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58971b-936c-469f-b679-3bf65286ea33 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.688345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3ac29-c214-4849-8e3b-ecadbc0a5739 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.691349 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1a58971b-936c-469f-b679-3bf65286ea33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:17.744814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed6ce4-6fac-4c7d-8339-a7f67bcbfad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.745882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed6ce4-6fac-4c7d-8339-a7f67bcbfad6 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.750853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed6ce4-6fac-4c7d-8339-a7f67bcbfad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.751295 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed6ce4-6fac-4c7d-8339-a7f67bcbfad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.753645 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-09ed6ce4-6fac-4c7d-8339-a7f67bcbfad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:17.768521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58971b-936c-469f-b679-3bf65286ea33 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.772866 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d4329-e7a1-4f88-b26d-2dfb4b474fd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.773557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d4329-e7a1-4f88-b26d-2dfb4b474fd4 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.774326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d4329-e7a1-4f88-b26d-2dfb4b474fd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.774901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d4329-e7a1-4f88-b26d-2dfb4b474fd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.793528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:17.833978 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed6ce4-6fac-4c7d-8339-a7f67bcbfad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.835472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed6ce4-6fac-4c7d-8339-a7f67bcbfad6 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.843668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d4329-e7a1-4f88-b26d-2dfb4b474fd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.844133 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 733/1480] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:17.845278 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d4329-e7a1-4f88-b26d-2dfb4b474fd4 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.846779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed6ce4-6fac-4c7d-8339-a7f67bcbfad6 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.855469 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 748/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:17.857034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed6ce4-6fac-4c7d-8339-a7f67bcbfad6 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.861817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3c61b-e354-4fd2-8745-bd7c9ad3dbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.862454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3c61b-e354-4fd2-8745-bd7c9ad3dbe2 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.862744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3c61b-e354-4fd2-8745-bd7c9ad3dbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.863002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3c61b-e354-4fd2-8745-bd7c9ad3dbe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.863640 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3c61b-e354-4fd2-8745-bd7c9ad3dbe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.865209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3c61b-e354-4fd2-8745-bd7c9ad3dbe2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.875457 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 734/1482] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:17.879712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d4329-e7a1-4f88-b26d-2dfb4b474fd4 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:17.892189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d4329-e7a1-4f88-b26d-2dfb4b474fd4 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:17.901640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242826-2f68-4f87-a262-814a27052d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:17.902834 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242826-2f68-4f87-a262-814a27052d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.903115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242826-2f68-4f87-a262-814a27052d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.903388 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242826-2f68-4f87-a262-814a27052d39 admin admin] PATH_INFO: `/v3/projects/c6558c9430f548e3bb14d86461d3df79/users/3727a5281e22483bb604ce15c9f6cdfa/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.905392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88242826-2f68-4f87-a262-814a27052d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6558c9430f548e3bb14d86461d3df79, user_id=3727a5281e22483bb604ce15c9f6cdfa, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:17.908580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88242826-2f68-4f87-a262-814a27052d39 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:17.920797 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 749/1483] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:17 2022] PUT /identity/v3/projects/c6558c9430f548e3bb14d86461d3df79/users/3727a5281e22483bb604ce15c9f6cdfa/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:17.932011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383cc22-c241-44b0-8c6a-ba3d9ece7b82 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:17.932213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383cc22-c241-44b0-8c6a-ba3d9ece7b82 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:17.932476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383cc22-c241-44b0-8c6a-ba3d9ece7b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:17.961292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1383cc22-c241-44b0-8c6a-ba3d9ece7b82 None None] MFA Rules not processed for user `3727a5281e22483bb604ce15c9f6cdfa`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:17.962599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1383cc22-c241-44b0-8c6a-ba3d9ece7b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:18.075010 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 735/1484] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:18.096762 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242826-2f68-4f87-a262-814a27052d39 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:18.098434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-88242826-2f68-4f87-a262-814a27052d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:18.162920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383cc22-c241-44b0-8c6a-ba3d9ece7b82 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:18.164596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1383cc22-c241-44b0-8c6a-ba3d9ece7b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:18.169876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242826-2f68-4f87-a262-814a27052d39 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:18.180500 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f53b4-3d0d-4682-89cf-4bd46caf8f76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:18.181260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f53b4-3d0d-4682-89cf-4bd46caf8f76 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:18.181552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f53b4-3d0d-4682-89cf-4bd46caf8f76 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:18.181825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f53b4-3d0d-4682-89cf-4bd46caf8f76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:18.200726 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f53b4-3d0d-4682-89cf-4bd46caf8f76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:18.202336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f53b4-3d0d-4682-89cf-4bd46caf8f76 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:18.212801 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 750/1485] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:18.279208 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383cc22-c241-44b0-8c6a-ba3d9ece7b82 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:18.289294 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:18.290345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:18.290628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:18.290942 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] PATH_INFO: `/v3/projects/aae17f90a690492b9921c73f7a6e5a76` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:18.291257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f53b4-3d0d-4682-89cf-4bd46caf8f76 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:18.292080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae17f90a690492b9921c73f7a6e5a76)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:18.293982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:18.299803 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f53b4-3d0d-4682-89cf-4bd46caf8f76 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:18.303981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22af5a7-2215-4a56-a9c0-9c05d75c204b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:18.304622 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22af5a7-2215-4a56-a9c0-9c05d75c204b service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:18.304853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22af5a7-2215-4a56-a9c0-9c05d75c204b service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:18.305127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22af5a7-2215-4a56-a9c0-9c05d75c204b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:18.307165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c22af5a7-2215-4a56-a9c0-9c05d75c204b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:18.312606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:18.352747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:18.354351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae17f90a690492b9921c73f7a6e5a76', 'request_id': 'req-be619238-ddee-4b87-9fa9-6d563f131db3'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:18.362026 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae17f90a690492b9921c73f7a6e5a76', 'request_id': 'req-be619238-ddee-4b87-9fa9-6d563f131db3'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:18.368963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae17f90a690492b9921c73f7a6e5a76', 'request_id': 'req-be619238-ddee-4b87-9fa9-6d563f131db3'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:18.375196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22af5a7-2215-4a56-a9c0-9c05d75c204b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:18.377010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22af5a7-2215-4a56-a9c0-9c05d75c204b service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:18.381461 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] The token cache is being invalidate because project aae17f90a690492b9921c73f7a6e5a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:18.381752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:18.382380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-be619238-ddee-4b87-9fa9-6d563f131db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:18.383925 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 736/1486] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:18 2022] DELETE /identity/v3/projects/aae17f90a690492b9921c73f7a6e5a76 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:18.386857 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 751/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:48:18.388678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17a78e-f8e6-4281-bb06-fb9d25e783b7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:18.388957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17a78e-f8e6-4281-bb06-fb9d25e783b7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:18.389252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17a78e-f8e6-4281-bb06-fb9d25e783b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:18.408809 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-df17a78e-f8e6-4281-bb06-fb9d25e783b7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:18.410023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-df17a78e-f8e6-4281-bb06-fb9d25e783b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:18.521017 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 737/1488] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:18.523098 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a9bf-1c44-4a3a-bc26-5a419291bf47 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:18.523556 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a9bf-1c44-4a3a-bc26-5a419291bf47 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:18.523830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a9bf-1c44-4a3a-bc26-5a419291bf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:18.552761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7087a9bf-1c44-4a3a-bc26-5a419291bf47 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:18.554112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7087a9bf-1c44-4a3a-bc26-5a419291bf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:18.567589 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 752/1489] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:18.572732 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17a78e-f8e6-4281-bb06-fb9d25e783b7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:18.597864 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17a78e-f8e6-4281-bb06-fb9d25e783b7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:18.613725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191aaa2-2570-4399-8a9f-0abc0c53e024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:18.614615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191aaa2-2570-4399-8a9f-0abc0c53e024 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:18.614930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191aaa2-2570-4399-8a9f-0abc0c53e024 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:18.615200 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191aaa2-2570-4399-8a9f-0abc0c53e024 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:18.615868 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191aaa2-2570-4399-8a9f-0abc0c53e024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:18.616174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191aaa2-2570-4399-8a9f-0abc0c53e024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:18.617872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191aaa2-2570-4399-8a9f-0abc0c53e024 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:18.647857 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 738/1490] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:18.652305 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a9bf-1c44-4a3a-bc26-5a419291bf47 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:18.666186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a9bf-1c44-4a3a-bc26-5a419291bf47 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:18.673275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e199c-e6c9-4e23-8a92-21d4c2781253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:18.674410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e199c-e6c9-4e23-8a92-21d4c2781253 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:18.675347 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e199c-e6c9-4e23-8a92-21d4c2781253 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:18.675919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e199c-e6c9-4e23-8a92-21d4c2781253 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:18.677035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e199c-e6c9-4e23-8a92-21d4c2781253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:18.680420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e199c-e6c9-4e23-8a92-21d4c2781253 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:18.739767 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 753/1491] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:18 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:18.743595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191aaa2-2570-4399-8a9f-0abc0c53e024 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:18.759546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191aaa2-2570-4399-8a9f-0abc0c53e024 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:18.932457 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc85985-0b32-404b-9a2f-2e6d61cbacc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:18.941944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc85985-0b32-404b-9a2f-2e6d61cbacc3 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:18.942676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc85985-0b32-404b-9a2f-2e6d61cbacc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:18.943069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc85985-0b32-404b-9a2f-2e6d61cbacc3 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:18.944009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc85985-0b32-404b-9a2f-2e6d61cbacc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:18.965181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc85985-0b32-404b-9a2f-2e6d61cbacc3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:18.993138 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 739/1492] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:18 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:18.997657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e199c-e6c9-4e23-8a92-21d4c2781253 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:19.009271 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e199c-e6c9-4e23-8a92-21d4c2781253 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:19.016587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c010687-759d-40a2-ba47-801ae2e931b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:19.017520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c010687-759d-40a2-ba47-801ae2e931b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:19.017857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c010687-759d-40a2-ba47-801ae2e931b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:19.018119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c010687-759d-40a2-ba47-801ae2e931b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:19.019387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c010687-759d-40a2-ba47-801ae2e931b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:19.020560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c010687-759d-40a2-ba47-801ae2e931b0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:19.034282 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 754/1493] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:19.036839 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc85985-0b32-404b-9a2f-2e6d61cbacc3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:19.049051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc85985-0b32-404b-9a2f-2e6d61cbacc3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:19.054297 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401a2d3-4429-424d-9a93-5e5639187a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:19.054939 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401a2d3-4429-424d-9a93-5e5639187a91 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:19.055217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401a2d3-4429-424d-9a93-5e5639187a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:19.055513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401a2d3-4429-424d-9a93-5e5639187a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:19.056287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401a2d3-4429-424d-9a93-5e5639187a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:19.057925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401a2d3-4429-424d-9a93-5e5639187a91 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:19.073929 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 740/1494] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:19.075772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c010687-759d-40a2-ba47-801ae2e931b0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:19.088418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c010687-759d-40a2-ba47-801ae2e931b0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:19.099453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcbfd2-23a7-463c-a578-3d6c693adead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:19.100120 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcbfd2-23a7-463c-a578-3d6c693adead admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:19.100445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcbfd2-23a7-463c-a578-3d6c693adead admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:19.100733 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcbfd2-23a7-463c-a578-3d6c693adead admin admin] PATH_INFO: `/v3/projects/0498a8458fba445198f5a4f3a706fd88/users/93918ababe02454d9c4f4767d98a4897/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:19.117201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdcbfd2-23a7-463c-a578-3d6c693adead admin admin] RBAC: Authorizing `identity:create_grant(project_id=0498a8458fba445198f5a4f3a706fd88, user_id=93918ababe02454d9c4f4767d98a4897, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:19.119791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdcbfd2-23a7-463c-a578-3d6c693adead admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:19.165271 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 755/1495] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:19 2022] PUT /identity/v3/projects/0498a8458fba445198f5a4f3a706fd88/users/93918ababe02454d9c4f4767d98a4897/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:19.175699 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45aa1d-cbb3-49f8-85ab-daf33c9042a1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:19.176058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45aa1d-cbb3-49f8-85ab-daf33c9042a1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:19.176403 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45aa1d-cbb3-49f8-85ab-daf33c9042a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:19.230296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1b45aa1d-cbb3-49f8-85ab-daf33c9042a1 None None] MFA Rules not processed for user `93918ababe02454d9c4f4767d98a4897`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:19.231954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1b45aa1d-cbb3-49f8-85ab-daf33c9042a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:19.277352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:19.334755 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 741/1496] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:19.344169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcbfd2-23a7-463c-a578-3d6c693adead admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:19.363809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6cdcbfd2-23a7-463c-a578-3d6c693adead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:19.511100 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcbfd2-23a7-463c-a578-3d6c693adead admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:19.515379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3ba50-5c58-4801-90f2-8673f59b0278 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b1c2ea68e4741728469ab7cc542b6d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:19.516258 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3ba50-5c58-4801-90f2-8673f59b0278 service glance] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:19.516592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3ba50-5c58-4801-90f2-8673f59b0278 service glance] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:19.516800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3ba50-5c58-4801-90f2-8673f59b0278 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:19.526323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3ba50-5c58-4801-90f2-8673f59b0278 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:19.528871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3ba50-5c58-4801-90f2-8673f59b0278 service glance] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:19.543602 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 756/1497] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:20.712654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45aa1d-cbb3-49f8-85ab-daf33c9042a1 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:20.714187 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1b45aa1d-cbb3-49f8-85ab-daf33c9042a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:20.830117 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45aa1d-cbb3-49f8-85ab-daf33c9042a1 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:20.834590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:20.835272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:20.835548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:20.835982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] PATH_INFO: `/v3/users/5e4cace7e4f8479492f24b7d2c3ebc57` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:20.837107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4cace7e4f8479492f24b7d2c3ebc57)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:20.839067 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:20.898991 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4cace7e4f8479492f24b7d2c3ebc57'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:20.899935 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4cace7e4f8479492f24b7d2c3ebc57'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:20.912173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4cace7e4f8479492f24b7d2c3ebc57'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:20.925413 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4cace7e4f8479492f24b7d2c3ebc57'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:20.951985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4cace7e4f8479492f24b7d2c3ebc57'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:20.964902 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 742/1498] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:20 2022] DELETE /identity/v3/users/5e4cace7e4f8479492f24b7d2c3ebc57 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:20.966672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f95ce-64e0-480d-835f-9dc05157b471 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:20.967247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f95ce-64e0-480d-835f-9dc05157b471 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:20.967247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f95ce-64e0-480d-835f-9dc05157b471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:20.987775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1f9f95ce-64e0-480d-835f-9dc05157b471 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:20.989038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1f9f95ce-64e0-480d-835f-9dc05157b471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:21.013179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:21.064846 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 757/1499] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:21.075850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:21.078575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:21.147010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02f0b8-3131-4a92-a1e6-3f75b950554a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:21.154048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055146c-ddf4-4945-a9a3-f8de5efad7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:21.154751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055146c-ddf4-4945-a9a3-f8de5efad7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:21.155136 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055146c-ddf4-4945-a9a3-f8de5efad7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:21.155440 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055146c-ddf4-4945-a9a3-f8de5efad7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:21.166784 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:21.177238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055146c-ddf4-4945-a9a3-f8de5efad7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:21.178750 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055146c-ddf4-4945-a9a3-f8de5efad7e4 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:21.190048 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 743/1500] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:21.480921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f95ce-64e0-480d-835f-9dc05157b471 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:21.482517 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1f9f95ce-64e0-480d-835f-9dc05157b471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:21.501618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f95ce-64e0-480d-835f-9dc05157b471 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:21.507036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:21.507689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:21.507964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:21.508238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] PATH_INFO: `/v3/projects/9537d272723b4d4290bbcbdcaf5d9235` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:21.509361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9537d272723b4d4290bbcbdcaf5d9235)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:21.511343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:21.575321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9537d272723b4d4290bbcbdcaf5d9235', 'request_id': 'req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.581286 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9537d272723b4d4290bbcbdcaf5d9235', 'request_id': 'req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.587960 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9537d272723b4d4290bbcbdcaf5d9235', 'request_id': 'req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.603382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] The token cache is being invalidate because project 9537d272723b4d4290bbcbdcaf5d9235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:21.603598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.604313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.606134 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 758/1501] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:21 2022] DELETE /identity/v3/projects/9537d272723b4d4290bbcbdcaf5d9235 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:21.610132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055146c-ddf4-4945-a9a3-f8de5efad7e4 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:21.620114 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055146c-ddf4-4945-a9a3-f8de5efad7e4 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:21.694744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:21.696401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:21.696401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:21.696401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] PATH_INFO: `/v3/users/e71c7541273442a0b3cb2e9bde43bd00` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:21.697105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71c7541273442a0b3cb2e9bde43bd00)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:21.699047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:21.770866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71c7541273442a0b3cb2e9bde43bd00'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.771924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71c7541273442a0b3cb2e9bde43bd00'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.791105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71c7541273442a0b3cb2e9bde43bd00'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.812127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71c7541273442a0b3cb2e9bde43bd00'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.828291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1b488d93-238d-4e07-a131-1de90bb33f58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71c7541273442a0b3cb2e9bde43bd00'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:21.834926 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 744/1502] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:21 2022] DELETE /identity/v3/users/e71c7541273442a0b3cb2e9bde43bd00 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:22.155030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:22.156430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:22.176942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b5a23-3c0f-4d9e-bdce-95b182ad1ef6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:22.283927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:22.284531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:22.284816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:22.285127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] PATH_INFO: `/v3/projects/2e2e281a16024323af5bc4dde1405539` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:22.286556 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2e281a16024323af5bc4dde1405539)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:22.288704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:22.344974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2e281a16024323af5bc4dde1405539', 'request_id': 'req-b93edb96-36a8-4dcb-897a-1a982bd53862'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.347448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:22.359320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2e281a16024323af5bc4dde1405539', 'request_id': 'req-b93edb96-36a8-4dcb-897a-1a982bd53862'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.370563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2e281a16024323af5bc4dde1405539', 'request_id': 'req-b93edb96-36a8-4dcb-897a-1a982bd53862'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.394582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] The token cache is being invalidate because project 2e2e281a16024323af5bc4dde1405539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:22.395639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.397787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.400090 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 759/1503] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:22 2022] DELETE /identity/v3/projects/2e2e281a16024323af5bc4dde1405539 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:22.404676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e906650-3660-4251-8ce8-03818ad10c53 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:22.404964 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e906650-3660-4251-8ce8-03818ad10c53 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:22.405239 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e906650-3660-4251-8ce8-03818ad10c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:22.433331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:22.435557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:22.451845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8e906650-3660-4251-8ce8-03818ad10c53 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:22.453381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8e906650-3660-4251-8ce8-03818ad10c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:22.562176 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 745/1504] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:22.571574 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e17ad-907a-4e25-8dca-86a3159367c8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:22.571995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e17ad-907a-4e25-8dca-86a3159367c8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:22.572276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e17ad-907a-4e25-8dca-86a3159367c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:22.603944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ad1e17ad-907a-4e25-8dca-86a3159367c8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:22.605685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ad1e17ad-907a-4e25-8dca-86a3159367c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:22.614559 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93edb96-36a8-4dcb-897a-1a982bd53862 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:22.643895 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 746/1505] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:22.652976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:22.653865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:22.654247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:22.654619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] PATH_INFO: `/v3/users/81deee2e9696413a86cd88af575dffaa` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:22.655988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81deee2e9696413a86cd88af575dffaa)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:22.660234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e17ad-907a-4e25-8dca-86a3159367c8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:22.661166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:22.677810 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e17ad-907a-4e25-8dca-86a3159367c8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:22.694471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdabf22-d007-47b8-99db-d0b0535dcc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:22.695140 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdabf22-d007-47b8-99db-d0b0535dcc90 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:22.695415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdabf22-d007-47b8-99db-d0b0535dcc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:22.695783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdabf22-d007-47b8-99db-d0b0535dcc90 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:22.696435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdabf22-d007-47b8-99db-d0b0535dcc90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:22.696733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdabf22-d007-47b8-99db-d0b0535dcc90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:22.699902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdabf22-d007-47b8-99db-d0b0535dcc90 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:22.709902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:22.734568 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 747/1506] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:22.745906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdabf22-d007-47b8-99db-d0b0535dcc90 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:22.773118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81deee2e9696413a86cd88af575dffaa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.773618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdabf22-d007-47b8-99db-d0b0535dcc90 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:22.785378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959e490-d70a-437e-8242-6c4bbd21aba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:22.786679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e490-d70a-437e-8242-6c4bbd21aba3 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:22.786994 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e490-d70a-437e-8242-6c4bbd21aba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:22.787270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959e490-d70a-437e-8242-6c4bbd21aba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:22.787912 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959e490-d70a-437e-8242-6c4bbd21aba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:22.790538 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959e490-d70a-437e-8242-6c4bbd21aba3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:22.797820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81deee2e9696413a86cd88af575dffaa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.798809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81deee2e9696413a86cd88af575dffaa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.817034 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81deee2e9696413a86cd88af575dffaa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.840071 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f4b86afd-c8ab-45b1-ba77-c41e423a3a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81deee2e9696413a86cd88af575dffaa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:22.858741 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 760/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:22 2022] DELETE /identity/v3/users/81deee2e9696413a86cd88af575dffaa => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:22.860056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ff175-2198-4cd3-a126-355190a5edac None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:22.860345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ff175-2198-4cd3-a126-355190a5edac None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:22.860620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ff175-2198-4cd3-a126-355190a5edac None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:22.861870 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 761/1508] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:48:22.863098 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab014a-7598-4563-98b4-e9c6ecde08f2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:22.863376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab014a-7598-4563-98b4-e9c6ecde08f2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:22.863650 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab014a-7598-4563-98b4-e9c6ecde08f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:22.868605 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 748/1509] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:22 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:22.869781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959e490-d70a-437e-8242-6c4bbd21aba3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:22.872619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1959e490-d70a-437e-8242-6c4bbd21aba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:22.894767 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a8ab014a-7598-4563-98b4-e9c6ecde08f2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:22.896183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a8ab014a-7598-4563-98b4-e9c6ecde08f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:23.001389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:23.024779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959e490-d70a-437e-8242-6c4bbd21aba3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.025705 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 762/1510] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:23.027159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4d3cf-848a-4d06-b1c6-13c149b56110 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.027751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4d3cf-848a-4d06-b1c6-13c149b56110 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.027751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4d3cf-848a-4d06-b1c6-13c149b56110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.036263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2daf437-e914-4117-b272-fc149d5844b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.036888 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daf437-e914-4117-b272-fc149d5844b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.037170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daf437-e914-4117-b272-fc149d5844b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.037442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daf437-e914-4117-b272-fc149d5844b6 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.038128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2daf437-e914-4117-b272-fc149d5844b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.041060 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2daf437-e914-4117-b272-fc149d5844b6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.060912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f3d4d3cf-848a-4d06-b1c6-13c149b56110 None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:23.062911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f3d4d3cf-848a-4d06-b1c6-13c149b56110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:23.085741 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 749/1511] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:22 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:23.089410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2daf437-e914-4117-b272-fc149d5844b6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.104973 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2daf437-e914-4117-b272-fc149d5844b6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.113615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf3317-0911-41b4-8682-f5debff593d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.114459 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf3317-0911-41b4-8682-f5debff593d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.114822 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf3317-0911-41b4-8682-f5debff593d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.115098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf3317-0911-41b4-8682-f5debff593d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.116170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf3317-0911-41b4-8682-f5debff593d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.117807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf3317-0911-41b4-8682-f5debff593d5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.135618 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 750/1512] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:23.139165 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf3317-0911-41b4-8682-f5debff593d5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.148235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1ecf3317-0911-41b4-8682-f5debff593d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:23.207711 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 763/1513] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:48:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:23.208956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4d3cf-848a-4d06-b1c6-13c149b56110 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.234931 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4d3cf-848a-4d06-b1c6-13c149b56110 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.241113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43383923-d09b-4233-b045-46c5385871fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.241888 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43383923-d09b-4233-b045-46c5385871fe admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.242319 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43383923-d09b-4233-b045-46c5385871fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.242618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43383923-d09b-4233-b045-46c5385871fe admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.243324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43383923-d09b-4233-b045-46c5385871fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.245115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43383923-d09b-4233-b045-46c5385871fe admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.261392 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 764/1514] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:23.268067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43383923-d09b-4233-b045-46c5385871fe admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.289034 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43383923-d09b-4233-b045-46c5385871fe admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.304619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae3b03-e089-4297-a5d5-5bf477e0ad6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.316902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf3317-0911-41b4-8682-f5debff593d5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.317783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae3b03-e089-4297-a5d5-5bf477e0ad6f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.318134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae3b03-e089-4297-a5d5-5bf477e0ad6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.318527 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae3b03-e089-4297-a5d5-5bf477e0ad6f admin admin] PATH_INFO: `/v3/projects/8df524958cd94fd1b6cf1dbaaf5fa2d6/users/931571b7fd4d4ea28e9d987e79701b5c/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.327137 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e7eb8-53eb-46ba-9f7c-265cc7611fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.328170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e7eb8-53eb-46ba-9f7c-265cc7611fdd service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.328835 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e7eb8-53eb-46ba-9f7c-265cc7611fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.330548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e7eb8-53eb-46ba-9f7c-265cc7611fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.366346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e7eb8-53eb-46ba-9f7c-265cc7611fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.367525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae3b03-e089-4297-a5d5-5bf477e0ad6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df524958cd94fd1b6cf1dbaaf5fa2d6, user_id=931571b7fd4d4ea28e9d987e79701b5c, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.367847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e7eb8-53eb-46ba-9f7c-265cc7611fdd service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.371358 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae3b03-e089-4297-a5d5-5bf477e0ad6f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.386003 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 751/1515] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:23.393591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e7eb8-53eb-46ba-9f7c-265cc7611fdd service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.399207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-326e7eb8-53eb-46ba-9f7c-265cc7611fdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:23.410621 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 765/1516] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:23 2022] PUT /identity/v3/projects/8df524958cd94fd1b6cf1dbaaf5fa2d6/users/931571b7fd4d4ea28e9d987e79701b5c/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:23.412266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30e097-fd3d-43d0-931b-9565cc1ee1af None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.412987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30e097-fd3d-43d0-931b-9565cc1ee1af None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.412987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30e097-fd3d-43d0-931b-9565cc1ee1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.440090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e7eb8-53eb-46ba-9f7c-265cc7611fdd service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.464328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ab30e097-fd3d-43d0-931b-9565cc1ee1af None None] MFA Rules not processed for user `931571b7fd4d4ea28e9d987e79701b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:23.472697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ab30e097-fd3d-43d0-931b-9565cc1ee1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:23.535795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:23.569868 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 766/1517] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:23.570847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96a0b4-f106-497f-8700-aac861b17c60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.571569 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96a0b4-f106-497f-8700-aac861b17c60 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.571851 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96a0b4-f106-497f-8700-aac861b17c60 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.572128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96a0b4-f106-497f-8700-aac861b17c60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.574371 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30e097-fd3d-43d0-931b-9565cc1ee1af None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.589045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96a0b4-f106-497f-8700-aac861b17c60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.591124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96a0b4-f106-497f-8700-aac861b17c60 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.604713 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 752/1518] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:23.674284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:23.706673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30e097-fd3d-43d0-931b-9565cc1ee1af None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.712781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e4152-a5f3-4594-9008-013316fcec80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.713530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e4152-a5f3-4594-9008-013316fcec80 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.713824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e4152-a5f3-4594-9008-013316fcec80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.714100 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e4152-a5f3-4594-9008-013316fcec80 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.715783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e4152-a5f3-4594-9008-013316fcec80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.720045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e4152-a5f3-4594-9008-013316fcec80 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.761053 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 767/1519] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:23 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:48:23.766041 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96a0b4-f106-497f-8700-aac861b17c60 service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.778606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96a0b4-f106-497f-8700-aac861b17c60 service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.844396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144f605-b7ec-406e-9493-e93ba6e0154b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.845079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144f605-b7ec-406e-9493-e93ba6e0154b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.845424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144f605-b7ec-406e-9493-e93ba6e0154b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.845761 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144f605-b7ec-406e-9493-e93ba6e0154b admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.846516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144f605-b7ec-406e-9493-e93ba6e0154b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.848567 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144f605-b7ec-406e-9493-e93ba6e0154b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.875229 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 753/1520] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:23 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:23.876821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e4152-a5f3-4594-9008-013316fcec80 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.893410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e4152-a5f3-4594-9008-013316fcec80 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.898688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec334d5-c773-41a2-be3b-f3703496556f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.899264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec334d5-c773-41a2-be3b-f3703496556f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.899537 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec334d5-c773-41a2-be3b-f3703496556f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.899805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec334d5-c773-41a2-be3b-f3703496556f admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.900443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec334d5-c773-41a2-be3b-f3703496556f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.902025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec334d5-c773-41a2-be3b-f3703496556f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.916036 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 768/1521] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:23.917315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144f605-b7ec-406e-9493-e93ba6e0154b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.928413 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144f605-b7ec-406e-9493-e93ba6e0154b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.933098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef6bef-4489-495b-8cb9-1a255353506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.933705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef6bef-4489-495b-8cb9-1a255353506e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.933979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef6bef-4489-495b-8cb9-1a255353506e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.934242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef6bef-4489-495b-8cb9-1a255353506e admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.934906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef6bef-4489-495b-8cb9-1a255353506e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.936483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef6bef-4489-495b-8cb9-1a255353506e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:23.947166 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 754/1522] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:23.953233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec334d5-c773-41a2-be3b-f3703496556f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:23.963494 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec334d5-c773-41a2-be3b-f3703496556f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:23.968400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbbc85-6a07-469a-9fae-ba649be88a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:23.969064 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbbc85-6a07-469a-9fae-ba649be88a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:23.969336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbbc85-6a07-469a-9fae-ba649be88a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:23.969862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbbc85-6a07-469a-9fae-ba649be88a4f admin admin] PATH_INFO: `/v3/projects/6ddfd05ed312444981a7cdd3eb799662/users/f6de67b8191849cb8d5e30636c87528d/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:23.986944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbbc85-6a07-469a-9fae-ba649be88a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ddfd05ed312444981a7cdd3eb799662, user_id=f6de67b8191849cb8d5e30636c87528d, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:23.989773 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbbc85-6a07-469a-9fae-ba649be88a4f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:24.008723 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 769/1523] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:23 2022] PUT /identity/v3/projects/6ddfd05ed312444981a7cdd3eb799662/users/f6de67b8191849cb8d5e30636c87528d/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:24.019745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb197ce5-eaec-4423-8098-1d70c906b174 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.020313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb197ce5-eaec-4423-8098-1d70c906b174 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.021493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb197ce5-eaec-4423-8098-1d70c906b174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:24.072437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-fb197ce5-eaec-4423-8098-1d70c906b174 None None] MFA Rules not processed for user `f6de67b8191849cb8d5e30636c87528d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:24.074542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fb197ce5-eaec-4423-8098-1d70c906b174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:24.154118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbbc85-6a07-469a-9fae-ba649be88a4f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:24.158853 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 755/1524] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:24.159244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-35fbbc85-6a07-469a-9fae-ba649be88a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:24.159880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb197ce5-eaec-4423-8098-1d70c906b174 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:24.252823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:24.277550 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbbc85-6a07-469a-9fae-ba649be88a4f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:24.284840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:24.285542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.285820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.286238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] PATH_INFO: `/v3/projects/2dcb344dd87642e89b540eac22f9d3c0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:24.287557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dcb344dd87642e89b540eac22f9d3c0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:24.293404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:24.304575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb197ce5-eaec-4423-8098-1d70c906b174 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:24.320423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c707f-27b0-4319-af1e-de17137737c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:24.321420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c707f-27b0-4319-af1e-de17137737c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.321899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c707f-27b0-4319-af1e-de17137737c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.322516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c707f-27b0-4319-af1e-de17137737c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:24.324404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c707f-27b0-4319-af1e-de17137737c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:24.329244 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c707f-27b0-4319-af1e-de17137737c8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:24.397854 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 756/1525] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:24 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:24.399209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c707f-27b0-4319-af1e-de17137737c8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:24.403166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dcb344dd87642e89b540eac22f9d3c0', 'request_id': 'req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.412775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dcb344dd87642e89b540eac22f9d3c0', 'request_id': 'req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.420350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dcb344dd87642e89b540eac22f9d3c0', 'request_id': 'req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.457746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] The token cache is being invalidate because project 2dcb344dd87642e89b540eac22f9d3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:24.458459 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.459168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.463991 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 770/1526] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:24 2022] DELETE /identity/v3/projects/2dcb344dd87642e89b540eac22f9d3c0 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:24.464896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:24.466350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:24.521141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:24.552352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c707f-27b0-4319-af1e-de17137737c8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:24.567384 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abec141-ca2e-4c02-ad04-cfadf34c10fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:24.568113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abec141-ca2e-4c02-ad04-cfadf34c10fe admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.568506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abec141-ca2e-4c02-ad04-cfadf34c10fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.568907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abec141-ca2e-4c02-ad04-cfadf34c10fe admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:24.570058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abec141-ca2e-4c02-ad04-cfadf34c10fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:24.572228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abec141-ca2e-4c02-ad04-cfadf34c10fe admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:24.593276 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fbdcb-1650-4d9e-b6ef-3b92f7542e30 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:24.599899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:24.600864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.601343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.601832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] PATH_INFO: `/v3/users/3727a5281e22483bb604ce15c9f6cdfa` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:24.603102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3727a5281e22483bb604ce15c9f6cdfa)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:24.605110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:24.607632 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 757/1527] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:24 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:24.611641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abec141-ca2e-4c02-ad04-cfadf34c10fe admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:24.614071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5abec141-ca2e-4c02-ad04-cfadf34c10fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:24.633866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abec141-ca2e-4c02-ad04-cfadf34c10fe admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:24.641809 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ae19b-c2b3-4a29-9ca8-b875d06d64ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:24.643925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ae19b-c2b3-4a29-9ca8-b875d06d64ec admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.644247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ae19b-c2b3-4a29-9ca8-b875d06d64ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.644618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ae19b-c2b3-4a29-9ca8-b875d06d64ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:24.645794 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ae19b-c2b3-4a29-9ca8-b875d06d64ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:24.648168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ae19b-c2b3-4a29-9ca8-b875d06d64ec admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:24.662181 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 758/1528] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:48:24.666937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ae19b-c2b3-4a29-9ca8-b875d06d64ec admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:24.695842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3727a5281e22483bb604ce15c9f6cdfa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.699357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ae19b-c2b3-4a29-9ca8-b875d06d64ec admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:24.707949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3727a5281e22483bb604ce15c9f6cdfa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.708697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3727a5281e22483bb604ce15c9f6cdfa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.711260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7c840-9902-43da-b5a1-fd1aadeba3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:24.713055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7c840-9902-43da-b5a1-fd1aadeba3ee admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.714072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7c840-9902-43da-b5a1-fd1aadeba3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.715883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7c840-9902-43da-b5a1-fd1aadeba3ee admin admin] PATH_INFO: `/v3/projects/1e82a86ff80d4fcb9c0ca76707f3d71b/users/3e2782d50c344e13b2b24accc214ac03/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:24.720480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3727a5281e22483bb604ce15c9f6cdfa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.740509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7c840-9902-43da-b5a1-fd1aadeba3ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e82a86ff80d4fcb9c0ca76707f3d71b, user_id=3e2782d50c344e13b2b24accc214ac03, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:24.743171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3727a5281e22483bb604ce15c9f6cdfa'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:24.744163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7c840-9902-43da-b5a1-fd1aadeba3ee admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:24.759178 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 771/1529] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:24 2022] DELETE /identity/v3/users/3727a5281e22483bb604ce15c9f6cdfa => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:24.771372 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 759/1530] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:24 2022] PUT /identity/v3/projects/1e82a86ff80d4fcb9c0ca76707f3d71b/users/3e2782d50c344e13b2b24accc214ac03/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:24.776104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:24.777530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:24.852626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02bcec-8618-4f35-b684-83197dda9554 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:24.902491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:24.919837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a812a-0748-4529-b39d-7d4c13bf1cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:24.920479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a812a-0748-4529-b39d-7d4c13bf1cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.920759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a812a-0748-4529-b39d-7d4c13bf1cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.921140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a812a-0748-4529-b39d-7d4c13bf1cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:24.921740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a812a-0748-4529-b39d-7d4c13bf1cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:24.923321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a812a-0748-4529-b39d-7d4c13bf1cb6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:24.933959 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 772/1531] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:24.938270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7c840-9902-43da-b5a1-fd1aadeba3ee admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:24.950182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7c840-9902-43da-b5a1-fd1aadeba3ee admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:24.956878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d5bc-b72c-4ac9-a560-ff8a7a30c673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:24.957765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d5bc-b72c-4ac9-a560-ff8a7a30c673 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.958079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d5bc-b72c-4ac9-a560-ff8a7a30c673 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.958395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d5bc-b72c-4ac9-a560-ff8a7a30c673 admin admin] PATH_INFO: `/v3/projects/1e82a86ff80d4fcb9c0ca76707f3d71b/users/3e2782d50c344e13b2b24accc214ac03/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:24.960332 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6d5bc-b72c-4ac9-a560-ff8a7a30c673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e82a86ff80d4fcb9c0ca76707f3d71b, user_id=3e2782d50c344e13b2b24accc214ac03, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:24.962984 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6d5bc-b72c-4ac9-a560-ff8a7a30c673 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:24.974661 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 760/1532] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:24 2022] PUT /identity/v3/projects/1e82a86ff80d4fcb9c0ca76707f3d71b/users/3e2782d50c344e13b2b24accc214ac03/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:24.986730 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907153f3-06a0-4c2f-98ef-1aa358647315 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:24.987035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907153f3-06a0-4c2f-98ef-1aa358647315 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:24.987384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907153f3-06a0-4c2f-98ef-1aa358647315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:25.016139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-907153f3-06a0-4c2f-98ef-1aa358647315 None None] MFA Rules not processed for user `3e2782d50c344e13b2b24accc214ac03`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:25.017403 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-907153f3-06a0-4c2f-98ef-1aa358647315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:25.054701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d5bc-b72c-4ac9-a560-ff8a7a30c673 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:25.058803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7fb6d5bc-b72c-4ac9-a560-ff8a7a30c673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:25.155265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d5bc-b72c-4ac9-a560-ff8a7a30c673 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:25.166108 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 773/1533] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:25.166849 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:25.168165 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:25.168738 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:25.169607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] PATH_INFO: `/v3/projects/c6558c9430f548e3bb14d86461d3df79` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:25.171237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6558c9430f548e3bb14d86461d3df79)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:25.173193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:25.180396 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907153f3-06a0-4c2f-98ef-1aa358647315 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:25.182504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-907153f3-06a0-4c2f-98ef-1aa358647315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:25.217206 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:25.286792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6558c9430f548e3bb14d86461d3df79', 'request_id': 'req-65f8277b-c637-438b-886c-926902323c11'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:25.295042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907153f3-06a0-4c2f-98ef-1aa358647315 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:25.297234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6558c9430f548e3bb14d86461d3df79', 'request_id': 'req-65f8277b-c637-438b-886c-926902323c11'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:25.311049 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6558c9430f548e3bb14d86461d3df79', 'request_id': 'req-65f8277b-c637-438b-886c-926902323c11'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:25.345710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] The token cache is being invalidate because project c6558c9430f548e3bb14d86461d3df79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:25.347346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:25.349031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65f8277b-c637-438b-886c-926902323c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:25.354290 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 761/1534] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:25 2022] DELETE /identity/v3/projects/c6558c9430f548e3bb14d86461d3df79 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:25.356857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b83c8-9648-4f93-9973-5d3f96f1d856 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:25.357215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b83c8-9648-4f93-9973-5d3f96f1d856 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:25.357583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b83c8-9648-4f93-9973-5d3f96f1d856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:25.383793 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-107b83c8-9648-4f93-9973-5d3f96f1d856 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:25.385499 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-107b83c8-9648-4f93-9973-5d3f96f1d856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:25.419106 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b13c7-231b-4f9e-b45c-bbc0de35bbe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:25.419789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b13c7-231b-4f9e-b45c-bbc0de35bbe1 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:25.420138 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b13c7-231b-4f9e-b45c-bbc0de35bbe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:25.420486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b13c7-231b-4f9e-b45c-bbc0de35bbe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:25.425208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b69b13c7-231b-4f9e-b45c-bbc0de35bbe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:25.523815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:25.525698 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 762/1535] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:25.531256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f561305-c06a-41c0-891e-9f3f1d694364 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:25.532127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f561305-c06a-41c0-891e-9f3f1d694364 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:25.532808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f561305-c06a-41c0-891e-9f3f1d694364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:25.562697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b13c7-231b-4f9e-b45c-bbc0de35bbe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:25.568713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b13c7-231b-4f9e-b45c-bbc0de35bbe1 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:25.586813 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 774/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:25.602688 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1f561305-c06a-41c0-891e-9f3f1d694364 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:25.606233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1f561305-c06a-41c0-891e-9f3f1d694364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:25.631705 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 763/1537] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:25.635917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b13c7-231b-4f9e-b45c-bbc0de35bbe1 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:25.664139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b13c7-231b-4f9e-b45c-bbc0de35bbe1 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:25.679461 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f479f-2e51-4f2d-88ce-7f8e3734e144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:25.680028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f479f-2e51-4f2d-88ce-7f8e3734e144 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:25.680313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f479f-2e51-4f2d-88ce-7f8e3734e144 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:25.680589 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f479f-2e51-4f2d-88ce-7f8e3734e144 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:25.681535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f479f-2e51-4f2d-88ce-7f8e3734e144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:25.681851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f479f-2e51-4f2d-88ce-7f8e3734e144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:25.683743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f479f-2e51-4f2d-88ce-7f8e3734e144 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:25.718457 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 775/1538] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:25.722607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f561305-c06a-41c0-891e-9f3f1d694364 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:25.752706 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f561305-c06a-41c0-891e-9f3f1d694364 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:25.763521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651787eb-971a-4b73-a708-afe5d20cf0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:25.764703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651787eb-971a-4b73-a708-afe5d20cf0eb admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:25.765209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651787eb-971a-4b73-a708-afe5d20cf0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:25.765812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651787eb-971a-4b73-a708-afe5d20cf0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:25.767043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651787eb-971a-4b73-a708-afe5d20cf0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:25.770037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651787eb-971a-4b73-a708-afe5d20cf0eb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:25.840681 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 764/1539] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:25 2022] POST /identity/v3/projects => generated 393 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:25.841532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f479f-2e51-4f2d-88ce-7f8e3734e144 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:25.863395 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f479f-2e51-4f2d-88ce-7f8e3734e144 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:25.965368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddd424-c925-480b-bb31-6bca4fd00c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:25.966379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddd424-c925-480b-bb31-6bca4fd00c58 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:25.966928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddd424-c925-480b-bb31-6bca4fd00c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:25.967615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddd424-c925-480b-bb31-6bca4fd00c58 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:25.968563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddd424-c925-480b-bb31-6bca4fd00c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:25.971785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddd424-c925-480b-bb31-6bca4fd00c58 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:26.001517 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 776/1540] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:25 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:26.009178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651787eb-971a-4b73-a708-afe5d20cf0eb admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:26.031651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651787eb-971a-4b73-a708-afe5d20cf0eb admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:26.040666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9c4b4-37ae-4098-8769-8f2faf4d556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:26.041699 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9c4b4-37ae-4098-8769-8f2faf4d556a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.042128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9c4b4-37ae-4098-8769-8f2faf4d556a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.042491 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9c4b4-37ae-4098-8769-8f2faf4d556a admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.043391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9c4b4-37ae-4098-8769-8f2faf4d556a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:26.045414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9c4b4-37ae-4098-8769-8f2faf4d556a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:26.060263 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 765/1541] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:26.065408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddd424-c925-480b-bb31-6bca4fd00c58 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:26.081894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddd424-c925-480b-bb31-6bca4fd00c58 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:26.090611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde6bfa-87c7-4116-8660-2f6a73f243ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:26.091496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde6bfa-87c7-4116-8660-2f6a73f243ce admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.092616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde6bfa-87c7-4116-8660-2f6a73f243ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.093087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde6bfa-87c7-4116-8660-2f6a73f243ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.094369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde6bfa-87c7-4116-8660-2f6a73f243ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:26.096430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde6bfa-87c7-4116-8660-2f6a73f243ce admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:26.131257 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 777/1542] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:48:26.135391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9c4b4-37ae-4098-8769-8f2faf4d556a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:26.150151 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9c4b4-37ae-4098-8769-8f2faf4d556a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:26.155972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569955f-4e11-4cd3-bd82-af54bc0ff912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:26.156839 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569955f-4e11-4cd3-bd82-af54bc0ff912 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.157328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569955f-4e11-4cd3-bd82-af54bc0ff912 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.157711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569955f-4e11-4cd3-bd82-af54bc0ff912 admin admin] PATH_INFO: `/v3/projects/258c1831489a4a53882f4609d60bc1d7/users/dbbfca34c2314913adf2562ee0172ed5/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.175774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569955f-4e11-4cd3-bd82-af54bc0ff912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=258c1831489a4a53882f4609d60bc1d7, user_id=dbbfca34c2314913adf2562ee0172ed5, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:26.179143 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569955f-4e11-4cd3-bd82-af54bc0ff912 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:26.196632 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 766/1543] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:26 2022] PUT /identity/v3/projects/258c1831489a4a53882f4609d60bc1d7/users/dbbfca34c2314913adf2562ee0172ed5/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:26.208804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b41cb-23c2-4d42-8b32-99847fd13998 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.209259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b41cb-23c2-4d42-8b32-99847fd13998 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.210287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b41cb-23c2-4d42-8b32-99847fd13998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.253700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2a4b41cb-23c2-4d42-8b32-99847fd13998 None None] MFA Rules not processed for user `dbbfca34c2314913adf2562ee0172ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:26.254984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2a4b41cb-23c2-4d42-8b32-99847fd13998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:26.372512 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 778/1544] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:26.376962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569955f-4e11-4cd3-bd82-af54bc0ff912 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:26.424950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:26.461679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569955f-4e11-4cd3-bd82-af54bc0ff912 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:26.468816 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf54da5-9d59-42d6-8ec1-3b866a6ce17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:26.469371 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf54da5-9d59-42d6-8ec1-3b866a6ce17d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.469703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf54da5-9d59-42d6-8ec1-3b866a6ce17d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.469917 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf54da5-9d59-42d6-8ec1-3b866a6ce17d admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.470697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf54da5-9d59-42d6-8ec1-3b866a6ce17d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:26.472724 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf54da5-9d59-42d6-8ec1-3b866a6ce17d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:26.520409 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 767/1545] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:26 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:26.524605 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b41cb-23c2-4d42-8b32-99847fd13998 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:26.540806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b41cb-23c2-4d42-8b32-99847fd13998 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:26.603532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:26.652851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2518633-c7df-4759-b011-3a2a674826f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:26.653550 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2518633-c7df-4759-b011-3a2a674826f3 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.653834 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2518633-c7df-4759-b011-3a2a674826f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.654108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2518633-c7df-4759-b011-3a2a674826f3 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.655786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2518633-c7df-4759-b011-3a2a674826f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:26.658379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2518633-c7df-4759-b011-3a2a674826f3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:26.710184 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 779/1546] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:26 2022] POST /identity/v3/users => generated 302 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:26.714098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf54da5-9d59-42d6-8ec1-3b866a6ce17d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:26.734849 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf54da5-9d59-42d6-8ec1-3b866a6ce17d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:26.741005 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a92c1-7a34-481f-9c9f-cf4871182691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:26.741840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a92c1-7a34-481f-9c9f-cf4871182691 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.743033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a92c1-7a34-481f-9c9f-cf4871182691 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.743033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a92c1-7a34-481f-9c9f-cf4871182691 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.743363 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a92c1-7a34-481f-9c9f-cf4871182691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:26.745180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a92c1-7a34-481f-9c9f-cf4871182691 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:26.763197 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 768/1547] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:26.764902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2518633-c7df-4759-b011-3a2a674826f3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:26.781260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2518633-c7df-4759-b011-3a2a674826f3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:26.794147 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e1080-5064-404e-9b5d-4677e5dc96a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:26.794147 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e1080-5064-404e-9b5d-4677e5dc96a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.795402 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e1080-5064-404e-9b5d-4677e5dc96a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.795721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e1080-5064-404e-9b5d-4677e5dc96a0 admin admin] PATH_INFO: `/v3/projects/04fc48bb327c40ecb743b16d6b2fdc57/users/fc1ca439a8124690822bfdecad69d374/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.825951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e1080-5064-404e-9b5d-4677e5dc96a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fc48bb327c40ecb743b16d6b2fdc57, user_id=fc1ca439a8124690822bfdecad69d374, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:26.829101 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e1080-5064-404e-9b5d-4677e5dc96a0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:26.850237 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 780/1548] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:26 2022] PUT /identity/v3/projects/04fc48bb327c40ecb743b16d6b2fdc57/users/fc1ca439a8124690822bfdecad69d374/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:26.854595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a92c1-7a34-481f-9c9f-cf4871182691 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:26.873326 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a92c1-7a34-481f-9c9f-cf4871182691 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:26.939526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6b0bc-95ed-46f1-ad63-8d3e67f430b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:26.940160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6b0bc-95ed-46f1-ad63-8d3e67f430b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.940518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6b0bc-95ed-46f1-ad63-8d3e67f430b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.940817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6b0bc-95ed-46f1-ad63-8d3e67f430b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.941498 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6b0bc-95ed-46f1-ad63-8d3e67f430b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:26.943260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6b0bc-95ed-46f1-ad63-8d3e67f430b4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:26.954019 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 769/1549] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:26.959159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e1080-5064-404e-9b5d-4677e5dc96a0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:26.975894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e1080-5064-404e-9b5d-4677e5dc96a0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:26.984299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d20740-bf80-4226-a69d-1a0eb47f0718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:26.985190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20740-bf80-4226-a69d-1a0eb47f0718 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:26.985590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20740-bf80-4226-a69d-1a0eb47f0718 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:26.986179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20740-bf80-4226-a69d-1a0eb47f0718 admin admin] PATH_INFO: `/v3/projects/04fc48bb327c40ecb743b16d6b2fdc57/users/fc1ca439a8124690822bfdecad69d374/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:26.988959 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d20740-bf80-4226-a69d-1a0eb47f0718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fc48bb327c40ecb743b16d6b2fdc57, user_id=fc1ca439a8124690822bfdecad69d374, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:26.991802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d20740-bf80-4226-a69d-1a0eb47f0718 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:27.002379 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 781/1550] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:26 2022] PUT /identity/v3/projects/04fc48bb327c40ecb743b16d6b2fdc57/users/fc1ca439a8124690822bfdecad69d374/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:27.013817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608eb2f-b52e-4f38-a4d3-f7ac18a3d92f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:27.014333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608eb2f-b52e-4f38-a4d3-f7ac18a3d92f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:27.014426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608eb2f-b52e-4f38-a4d3-f7ac18a3d92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:27.045311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2608eb2f-b52e-4f38-a4d3-f7ac18a3d92f None None] MFA Rules not processed for user `fc1ca439a8124690822bfdecad69d374`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:27.046658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2608eb2f-b52e-4f38-a4d3-f7ac18a3d92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:27.123619 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 770/1551] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:27 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:27.135275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d20740-bf80-4226-a69d-1a0eb47f0718 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:27.137180 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-04d20740-bf80-4226-a69d-1a0eb47f0718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:27.196239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d20740-bf80-4226-a69d-1a0eb47f0718 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:27.200084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9ed49-63ca-43ed-97f2-4e4c19b26244 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:27.200700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9ed49-63ca-43ed-97f2-4e4c19b26244 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:27.200975 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9ed49-63ca-43ed-97f2-4e4c19b26244 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:27.201241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9ed49-63ca-43ed-97f2-4e4c19b26244 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:27.210603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9ed49-63ca-43ed-97f2-4e4c19b26244 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:27.212328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9ed49-63ca-43ed-97f2-4e4c19b26244 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:27.224187 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 782/1552] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:27 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:27.300771 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608eb2f-b52e-4f38-a4d3-f7ac18a3d92f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:27.314911 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608eb2f-b52e-4f38-a4d3-f7ac18a3d92f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:27.318492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6b743-7bba-41e2-b7f4-3bf19338a04a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:27.319154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6b743-7bba-41e2-b7f4-3bf19338a04a service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:27.319432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6b743-7bba-41e2-b7f4-3bf19338a04a service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:27.319703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6b743-7bba-41e2-b7f4-3bf19338a04a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:27.369094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:27.387742 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9ed49-63ca-43ed-97f2-4e4c19b26244 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:27.389688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-75e9ed49-63ca-43ed-97f2-4e4c19b26244 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:27.399082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6b743-7bba-41e2-b7f4-3bf19338a04a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:27.401430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6b743-7bba-41e2-b7f4-3bf19338a04a service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:27.406229 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9ed49-63ca-43ed-97f2-4e4c19b26244 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:27.408408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efccd9-042f-425d-9a73-fa2a17ed9df1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:27.409151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efccd9-042f-425d-9a73-fa2a17ed9df1 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:27.409491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efccd9-042f-425d-9a73-fa2a17ed9df1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:27.409826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efccd9-042f-425d-9a73-fa2a17ed9df1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:27.411635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-83efccd9-042f-425d-9a73-fa2a17ed9df1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:27.424189 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 771/1553] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:27.484882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efccd9-042f-425d-9a73-fa2a17ed9df1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:27.486246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efccd9-042f-425d-9a73-fa2a17ed9df1 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:27.496387 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 783/1554] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:48:28.221779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6b743-7bba-41e2-b7f4-3bf19338a04a service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:28.230391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6b743-7bba-41e2-b7f4-3bf19338a04a service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:28.233991 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09125c78-bd4d-4ede-9a7e-d9323915dadf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:28.234584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09125c78-bd4d-4ede-9a7e-d9323915dadf service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:28.234898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09125c78-bd4d-4ede-9a7e-d9323915dadf service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:28.235173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09125c78-bd4d-4ede-9a7e-d9323915dadf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:28.236819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-09125c78-bd4d-4ede-9a7e-d9323915dadf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:28.304988 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09125c78-bd4d-4ede-9a7e-d9323915dadf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:28.306437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09125c78-bd4d-4ede-9a7e-d9323915dadf service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:28.316193 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 772/1555] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:31.255230 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efccd9-042f-425d-9a73-fa2a17ed9df1 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:31.308506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:31.357942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efccd9-042f-425d-9a73-fa2a17ed9df1 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:31.360586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:31.361420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:31.361758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:31.362202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] PATH_INFO: `/v3/users/dbbfca34c2314913adf2562ee0172ed5` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:31.364185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbbfca34c2314913adf2562ee0172ed5)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:31.366612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:31.431700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbbfca34c2314913adf2562ee0172ed5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:31.443113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbbfca34c2314913adf2562ee0172ed5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:31.443113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbbfca34c2314913adf2562ee0172ed5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:31.450746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbbfca34c2314913adf2562ee0172ed5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:31.463407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbbfca34c2314913adf2562ee0172ed5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:31.474550 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 784/1556] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:31 2022] DELETE /identity/v3/users/dbbfca34c2314913adf2562ee0172ed5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:31.479314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53de081-cf4b-4d0d-8acd-db5ad071d2dd None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:31.479800 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53de081-cf4b-4d0d-8acd-db5ad071d2dd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:31.479885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53de081-cf4b-4d0d-8acd-db5ad071d2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:31.638135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c53de081-cf4b-4d0d-8acd-db5ad071d2dd None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:31.639478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c53de081-cf4b-4d0d-8acd-db5ad071d2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:31.720973 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 773/1557] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:31.731237 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:31.732908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:31.798609 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a39e7-91bc-449a-a67b-d4a554b70f79 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:31.805379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80446d-bec1-45fa-988b-3680896fe7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:31.806096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80446d-bec1-45fa-988b-3680896fe7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:31.806587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80446d-bec1-45fa-988b-3680896fe7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:31.806891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80446d-bec1-45fa-988b-3680896fe7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:31.818228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80446d-bec1-45fa-988b-3680896fe7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:31.820791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80446d-bec1-45fa-988b-3680896fe7f8 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:31.832448 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 785/1558] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:32.308950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53de081-cf4b-4d0d-8acd-db5ad071d2dd None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:32.310923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c53de081-cf4b-4d0d-8acd-db5ad071d2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:32.324095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53de081-cf4b-4d0d-8acd-db5ad071d2dd None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:32.328899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:32.329556 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:32.329833 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:32.330104 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] PATH_INFO: `/v3/projects/258c1831489a4a53882f4609d60bc1d7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:32.331290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=258c1831489a4a53882f4609d60bc1d7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:32.333215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:32.381767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258c1831489a4a53882f4609d60bc1d7', 'request_id': 'req-ab468f13-df4f-49b5-ab3c-8e93571b4f26'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.387969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258c1831489a4a53882f4609d60bc1d7', 'request_id': 'req-ab468f13-df4f-49b5-ab3c-8e93571b4f26'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.394449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258c1831489a4a53882f4609d60bc1d7', 'request_id': 'req-ab468f13-df4f-49b5-ab3c-8e93571b4f26'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.396136 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:32.409058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] The token cache is being invalidate because project 258c1831489a4a53882f4609d60bc1d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:32.409421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.410089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.411657 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 774/1559] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:32 2022] DELETE /identity/v3/projects/258c1831489a4a53882f4609d60bc1d7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:32.416260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80446d-bec1-45fa-988b-3680896fe7f8 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:32.430680 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80446d-bec1-45fa-988b-3680896fe7f8 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:32.513295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:32.520271 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:32.522112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:32.529950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:32.531022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:32.531456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:32.531868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] PATH_INFO: `/v3/users/fc1ca439a8124690822bfdecad69d374` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:32.533313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1ca439a8124690822bfdecad69d374)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:32.535960 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:32.598601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1ca439a8124690822bfdecad69d374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.604009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab468f13-df4f-49b5-ab3c-8e93571b4f26 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:32.610246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:32.611163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:32.611506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:32.611863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] PATH_INFO: `/v3/users/931571b7fd4d4ea28e9d987e79701b5c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:32.612958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1ca439a8124690822bfdecad69d374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.613720 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=931571b7fd4d4ea28e9d987e79701b5c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:32.614158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1ca439a8124690822bfdecad69d374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.616005 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:32.629288 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1ca439a8124690822bfdecad69d374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.648018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bf497081-c578-4177-b30b-8031eb6d1210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1ca439a8124690822bfdecad69d374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.661249 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 786/1560] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:32 2022] DELETE /identity/v3/users/fc1ca439a8124690822bfdecad69d374 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:32.681236 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '931571b7fd4d4ea28e9d987e79701b5c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.682177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '931571b7fd4d4ea28e9d987e79701b5c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.691439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '931571b7fd4d4ea28e9d987e79701b5c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.700305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '931571b7fd4d4ea28e9d987e79701b5c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.715159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '931571b7fd4d4ea28e9d987e79701b5c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:32.727079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ea7e6-c7e7-4148-9bf7-f6ef0790ed81 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:32.727368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ea7e6-c7e7-4148-9bf7-f6ef0790ed81 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:32.727643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ea7e6-c7e7-4148-9bf7-f6ef0790ed81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:32.730734 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 775/1561] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:32 2022] DELETE /identity/v3/users/931571b7fd4d4ea28e9d987e79701b5c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:32.749382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f64ea7e6-c7e7-4148-9bf7-f6ef0790ed81 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:32.750970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f64ea7e6-c7e7-4148-9bf7-f6ef0790ed81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:32.870811 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 787/1562] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:32.886645 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:32.888410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:32.955690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c437d-ccb7-40a7-b1b2-ec1d2dfdbb81 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:32.960247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4180c29-5d83-49a2-a401-208a3fcb1f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:32.960859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4180c29-5d83-49a2-a401-208a3fcb1f75 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:32.961139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4180c29-5d83-49a2-a401-208a3fcb1f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:32.962319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4180c29-5d83-49a2-a401-208a3fcb1f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:32.975554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4180c29-5d83-49a2-a401-208a3fcb1f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:32.977290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4180c29-5d83-49a2-a401-208a3fcb1f75 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:32.989277 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 776/1563] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:33.617086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ea7e6-c7e7-4148-9bf7-f6ef0790ed81 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:33.618596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f64ea7e6-c7e7-4148-9bf7-f6ef0790ed81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:33.630991 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ea7e6-c7e7-4148-9bf7-f6ef0790ed81 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:33.635700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:33.636338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:33.636613 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:33.636884 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] PATH_INFO: `/v3/projects/04fc48bb327c40ecb743b16d6b2fdc57` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:33.637986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04fc48bb327c40ecb743b16d6b2fdc57)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:33.639935 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:33.704887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04fc48bb327c40ecb743b16d6b2fdc57', 'request_id': 'req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:33.710700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04fc48bb327c40ecb743b16d6b2fdc57', 'request_id': 'req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:33.716852 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04fc48bb327c40ecb743b16d6b2fdc57', 'request_id': 'req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:33.720634 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:33.732569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] The token cache is being invalidate because project 04fc48bb327c40ecb743b16d6b2fdc57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:33.732831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:33.733460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ac0fb79a-2138-4315-a5d8-4f53fe47ea01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:33.735019 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 788/1564] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:33 2022] DELETE /identity/v3/projects/04fc48bb327c40ecb743b16d6b2fdc57 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:33.740060 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ada3e5-0c9b-4d96-93e6-b257c2b3c19e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:33.740560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ada3e5-0c9b-4d96-93e6-b257c2b3c19e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:33.740657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ada3e5-0c9b-4d96-93e6-b257c2b3c19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:33.760020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a3ada3e5-0c9b-4d96-93e6-b257c2b3c19e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:33.761296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a3ada3e5-0c9b-4d96-93e6-b257c2b3c19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:33.769747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:33.895694 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 777/1565] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:33.896558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdf1c7-bdb8-49d9-8a58-88856c29b5a6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:33.896558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdf1c7-bdb8-49d9-8a58-88856c29b5a6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:33.896558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdf1c7-bdb8-49d9-8a58-88856c29b5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:33.922751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-aacdf1c7-bdb8-49d9-8a58-88856c29b5a6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:33.924626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aacdf1c7-bdb8-49d9-8a58-88856c29b5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:33.934765 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 789/1566] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:33.938584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ada3e5-0c9b-4d96-93e6-b257c2b3c19e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:33.954150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ada3e5-0c9b-4d96-93e6-b257c2b3c19e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:33.960004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741507a-0524-49a5-93f7-cf1886103c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:33.960930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741507a-0524-49a5-93f7-cf1886103c02 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:33.961212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741507a-0524-49a5-93f7-cf1886103c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:33.961542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741507a-0524-49a5-93f7-cf1886103c02 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:33.962223 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741507a-0524-49a5-93f7-cf1886103c02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:33.962579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741507a-0524-49a5-93f7-cf1886103c02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:33.964373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741507a-0524-49a5-93f7-cf1886103c02 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:33.975534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdf1c7-bdb8-49d9-8a58-88856c29b5a6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:33.980817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aacdf1c7-bdb8-49d9-8a58-88856c29b5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:33.998416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdf1c7-bdb8-49d9-8a58-88856c29b5a6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.001556 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 778/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:34.007350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741507a-0524-49a5-93f7-cf1886103c02 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.008807 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.013242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.014331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.014606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] PATH_INFO: `/v3/projects/8df524958cd94fd1b6cf1dbaaf5fa2d6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.017584 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df524958cd94fd1b6cf1dbaaf5fa2d6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.030436 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.033076 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741507a-0524-49a5-93f7-cf1886103c02 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.047925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341026d-7e2c-402e-bd94-7a3fba3970d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.049126 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341026d-7e2c-402e-bd94-7a3fba3970d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.049616 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341026d-7e2c-402e-bd94-7a3fba3970d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.050036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341026d-7e2c-402e-bd94-7a3fba3970d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.050982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341026d-7e2c-402e-bd94-7a3fba3970d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.053475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341026d-7e2c-402e-bd94-7a3fba3970d5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.088184 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 779/1568] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:34 2022] POST /identity/v3/projects => generated 395 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:34.092246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341026d-7e2c-402e-bd94-7a3fba3970d5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.103646 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df524958cd94fd1b6cf1dbaaf5fa2d6', 'request_id': 'req-6207258d-ffdc-43ab-b2b3-cbea6c26883a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.109950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df524958cd94fd1b6cf1dbaaf5fa2d6', 'request_id': 'req-6207258d-ffdc-43ab-b2b3-cbea6c26883a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.117712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df524958cd94fd1b6cf1dbaaf5fa2d6', 'request_id': 'req-6207258d-ffdc-43ab-b2b3-cbea6c26883a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.136118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] The token cache is being invalidate because project 8df524958cd94fd1b6cf1dbaaf5fa2d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:34.136412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.137111 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.139316 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 790/1569] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:33 2022] DELETE /identity/v3/projects/8df524958cd94fd1b6cf1dbaaf5fa2d6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:34.144972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.147106 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:34.177989 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341026d-7e2c-402e-bd94-7a3fba3970d5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.184747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd712300-40f8-4e97-9851-d09301260ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.184941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd712300-40f8-4e97-9851-d09301260ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.185376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd712300-40f8-4e97-9851-d09301260ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.185907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd712300-40f8-4e97-9851-d09301260ad7 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.186661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd712300-40f8-4e97-9851-d09301260ad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.189321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd712300-40f8-4e97-9851-d09301260ad7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.213558 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 780/1570] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:34 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:34.217734 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd712300-40f8-4e97-9851-d09301260ad7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.219415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fd712300-40f8-4e97-9851-d09301260ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:34.229168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207258d-ffdc-43ab-b2b3-cbea6c26883a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.232324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd712300-40f8-4e97-9851-d09301260ad7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.234454 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.235191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.235448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.235722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] PATH_INFO: `/v3/users/f6de67b8191849cb8d5e30636c87528d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.236863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6de67b8191849cb8d5e30636c87528d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.237671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af080b07-67fe-47f4-858c-edece57eb8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.238377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af080b07-67fe-47f4-858c-edece57eb8ad admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.238793 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af080b07-67fe-47f4-858c-edece57eb8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.239158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af080b07-67fe-47f4-858c-edece57eb8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.239299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.239984 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af080b07-67fe-47f4-858c-edece57eb8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.241689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af080b07-67fe-47f4-858c-edece57eb8ad admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.254201 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 781/1571] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:48:34.258494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af080b07-67fe-47f4-858c-edece57eb8ad admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.275479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af080b07-67fe-47f4-858c-edece57eb8ad admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.285583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c5789-3ed2-45ff-912b-0585e713efa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.286285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c5789-3ed2-45ff-912b-0585e713efa1 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.286666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c5789-3ed2-45ff-912b-0585e713efa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.287089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c5789-3ed2-45ff-912b-0585e713efa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.287621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c5789-3ed2-45ff-912b-0585e713efa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.289456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c5789-3ed2-45ff-912b-0585e713efa1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.301933 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 782/1572] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:48:34.306125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c5789-3ed2-45ff-912b-0585e713efa1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.309479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6de67b8191849cb8d5e30636c87528d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.316636 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c5789-3ed2-45ff-912b-0585e713efa1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.317479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6de67b8191849cb8d5e30636c87528d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.318285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6de67b8191849cb8d5e30636c87528d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.321763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe743ff-cb67-48b7-ae89-7803004e8ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.322441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe743ff-cb67-48b7-ae89-7803004e8ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.322707 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe743ff-cb67-48b7-ae89-7803004e8ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.323009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe743ff-cb67-48b7-ae89-7803004e8ee0 admin admin] PATH_INFO: `/v3/projects/31049e0e2ef84586930562d555f1bcc3/users/71b24019a949461a8863db9dd70d93b2/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.328736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6de67b8191849cb8d5e30636c87528d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.338277 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:34.340085 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe743ff-cb67-48b7-ae89-7803004e8ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31049e0e2ef84586930562d555f1bcc3, user_id=71b24019a949461a8863db9dd70d93b2, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.343350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe743ff-cb67-48b7-ae89-7803004e8ee0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.344804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f17500f4-07ec-481f-ad5e-11e2c604eb4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6de67b8191849cb8d5e30636c87528d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.356329 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 791/1573] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:34 2022] DELETE /identity/v3/users/f6de67b8191849cb8d5e30636c87528d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:34.363362 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 783/1574] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:34 2022] PUT /identity/v3/projects/31049e0e2ef84586930562d555f1bcc3/users/71b24019a949461a8863db9dd70d93b2/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:34.374051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324a13c-c3c6-445a-8578-3791577d6db6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.374352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324a13c-c3c6-445a-8578-3791577d6db6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.374605 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324a13c-c3c6-445a-8578-3791577d6db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.407988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2324a13c-c3c6-445a-8578-3791577d6db6 None None] MFA Rules not processed for user `71b24019a949461a8863db9dd70d93b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:34.409472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2324a13c-c3c6-445a-8578-3791577d6db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:34.494746 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 792/1575] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:34 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:34.501023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe743ff-cb67-48b7-ae89-7803004e8ee0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.503599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0fe743ff-cb67-48b7-ae89-7803004e8ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:34.548006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:34.642524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe743ff-cb67-48b7-ae89-7803004e8ee0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.732073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569dbed-94b7-4693-8c54-5e01d04f8f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.732743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569dbed-94b7-4693-8c54-5e01d04f8f3c admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.733018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569dbed-94b7-4693-8c54-5e01d04f8f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.733285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569dbed-94b7-4693-8c54-5e01d04f8f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.733947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569dbed-94b7-4693-8c54-5e01d04f8f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.736813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569dbed-94b7-4693-8c54-5e01d04f8f3c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.758914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324a13c-c3c6-445a-8578-3791577d6db6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.761143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2324a13c-c3c6-445a-8578-3791577d6db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:34.782845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324a13c-c3c6-445a-8578-3791577d6db6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.783982 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 784/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:34 2022] POST /identity/v3/projects => generated 399 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:34.788032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569dbed-94b7-4693-8c54-5e01d04f8f3c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.864434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.865191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.865655 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.866017 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] PATH_INFO: `/v3/projects/6ddfd05ed312444981a7cdd3eb799662` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.867296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ddfd05ed312444981a7cdd3eb799662)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.869153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.874064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569dbed-94b7-4693-8c54-5e01d04f8f3c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.879947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54cbc3-23f9-42f5-8222-f173665dd197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:34.880563 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54cbc3-23f9-42f5-8222-f173665dd197 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:34.880857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54cbc3-23f9-42f5-8222-f173665dd197 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:34.881110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54cbc3-23f9-42f5-8222-f173665dd197 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:34.881742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54cbc3-23f9-42f5-8222-f173665dd197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:34.883798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54cbc3-23f9-42f5-8222-f173665dd197 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:34.911667 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 785/1577] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:34 2022] POST /identity/v3/users => generated 306 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:48:34.916023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54cbc3-23f9-42f5-8222-f173665dd197 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.926232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54cbc3-23f9-42f5-8222-f173665dd197 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:34.926698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ddfd05ed312444981a7cdd3eb799662', 'request_id': 'req-391579be-dbbb-4caf-b407-ebc59e26460c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.932943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ddfd05ed312444981a7cdd3eb799662', 'request_id': 'req-391579be-dbbb-4caf-b407-ebc59e26460c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.939887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ddfd05ed312444981a7cdd3eb799662', 'request_id': 'req-391579be-dbbb-4caf-b407-ebc59e26460c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.974428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] The token cache is being invalidate because project 6ddfd05ed312444981a7cdd3eb799662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:34.974743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.976299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:34.981603 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 793/1578] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:34 2022] DELETE /identity/v3/projects/6ddfd05ed312444981a7cdd3eb799662 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:34.981739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:34.982831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:34.985507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:35.026521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:35.044506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f623c-8e9a-4bb7-98f1-ce1d7bc14679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:35.045762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f623c-8e9a-4bb7-98f1-ce1d7bc14679 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:35.046552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f623c-8e9a-4bb7-98f1-ce1d7bc14679 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:35.046996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f623c-8e9a-4bb7-98f1-ce1d7bc14679 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:35.047752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f623c-8e9a-4bb7-98f1-ce1d7bc14679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:35.050614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f623c-8e9a-4bb7-98f1-ce1d7bc14679 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:35.083892 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 786/1579] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:48:35.089485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f623c-8e9a-4bb7-98f1-ce1d7bc14679 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:35.093355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9e2f623c-8e9a-4bb7-98f1-ce1d7bc14679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:35.120191 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f623c-8e9a-4bb7-98f1-ce1d7bc14679 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:35.130036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da011dc-1aa1-4232-88ba-5c3adf4acfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:35.131815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391579be-dbbb-4caf-b407-ebc59e26460c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:35.133068 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da011dc-1aa1-4232-88ba-5c3adf4acfe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:35.133450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da011dc-1aa1-4232-88ba-5c3adf4acfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:35.134242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da011dc-1aa1-4232-88ba-5c3adf4acfe6 admin admin] PATH_INFO: `/v3/projects/93e1ad5db0304facbcf9242bf687f347/users/e989127ad9514a24bb389ee3a5a9e63e/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:35.142404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:35.143089 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:35.143785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:35.144049 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] PATH_INFO: `/v3/users/3e2782d50c344e13b2b24accc214ac03` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:35.146131 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2782d50c344e13b2b24accc214ac03)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:35.148540 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:35.157657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da011dc-1aa1-4232-88ba-5c3adf4acfe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e1ad5db0304facbcf9242bf687f347, user_id=e989127ad9514a24bb389ee3a5a9e63e, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:35.160750 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da011dc-1aa1-4232-88ba-5c3adf4acfe6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:35.184087 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 787/1580] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:35 2022] PUT /identity/v3/projects/93e1ad5db0304facbcf9242bf687f347/users/e989127ad9514a24bb389ee3a5a9e63e/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:35.187682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da011dc-1aa1-4232-88ba-5c3adf4acfe6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:35.228418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2782d50c344e13b2b24accc214ac03'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:35.237257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2782d50c344e13b2b24accc214ac03'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:35.238040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2782d50c344e13b2b24accc214ac03'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:35.252910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2782d50c344e13b2b24accc214ac03'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:35.259102 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da011dc-1aa1-4232-88ba-5c3adf4acfe6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:35.264666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef8be2-4021-483a-9240-9f7bfa3e9ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:35.265379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef8be2-4021-483a-9240-9f7bfa3e9ddf admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:35.265831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef8be2-4021-483a-9240-9f7bfa3e9ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:35.266045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef8be2-4021-483a-9240-9f7bfa3e9ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:35.270238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2782d50c344e13b2b24accc214ac03'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:35.279088 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 794/1581] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:34 2022] DELETE /identity/v3/users/3e2782d50c344e13b2b24accc214ac03 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:35.291177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef8be2-4021-483a-9240-9f7bfa3e9ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:35.293744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef8be2-4021-483a-9240-9f7bfa3e9ddf admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:35.309201 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 788/1582] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:48:35.311359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:35.314795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:35.508352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4b801-a8f6-498c-b197-ec5d04072e09 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:35.521903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3b93d-bdcf-43ac-a090-0bcd46e7e6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:35.522613 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3b93d-bdcf-43ac-a090-0bcd46e7e6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:35.522919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3b93d-bdcf-43ac-a090-0bcd46e7e6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:35.523196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3b93d-bdcf-43ac-a090-0bcd46e7e6cd admin admin] PATH_INFO: `/v3/projects/93e1ad5db0304facbcf9242bf687f347/users/e989127ad9514a24bb389ee3a5a9e63e/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:35.526481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3b93d-bdcf-43ac-a090-0bcd46e7e6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e1ad5db0304facbcf9242bf687f347, user_id=e989127ad9514a24bb389ee3a5a9e63e, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:35.529429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3b93d-bdcf-43ac-a090-0bcd46e7e6cd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:35.559456 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 795/1583] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:35 2022] PUT /identity/v3/projects/93e1ad5db0304facbcf9242bf687f347/users/e989127ad9514a24bb389ee3a5a9e63e/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:35.560676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791789ab-e556-41cd-9213-ae26ba4d4b49 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:35.561228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791789ab-e556-41cd-9213-ae26ba4d4b49 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:35.561573 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791789ab-e556-41cd-9213-ae26ba4d4b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:35.630254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-791789ab-e556-41cd-9213-ae26ba4d4b49 None None] MFA Rules not processed for user `e989127ad9514a24bb389ee3a5a9e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:35.634293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-791789ab-e556-41cd-9213-ae26ba4d4b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:35.756547 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 789/1584] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:35 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:35.767860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3b93d-bdcf-43ac-a090-0bcd46e7e6cd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:35.770374 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-64c3b93d-bdcf-43ac-a090-0bcd46e7e6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:35.827621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:35.869315 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3b93d-bdcf-43ac-a090-0bcd46e7e6cd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:35.873342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57354e1-5885-4a6d-b8ac-7b212317b40c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:35.874515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57354e1-5885-4a6d-b8ac-7b212317b40c service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:35.874515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57354e1-5885-4a6d-b8ac-7b212317b40c service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:35.874755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57354e1-5885-4a6d-b8ac-7b212317b40c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:35.886380 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57354e1-5885-4a6d-b8ac-7b212317b40c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:35.887132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57354e1-5885-4a6d-b8ac-7b212317b40c service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:35.898208 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 796/1585] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:35 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:35.937362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791789ab-e556-41cd-9213-ae26ba4d4b49 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:35.939076 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-791789ab-e556-41cd-9213-ae26ba4d4b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:36.021165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57354e1-5885-4a6d-b8ac-7b212317b40c service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:36.023820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:36.032582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57354e1-5885-4a6d-b8ac-7b212317b40c service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:36.036671 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4871d-130d-4efd-8a4f-37708f82e408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:36.037394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4871d-130d-4efd-8a4f-37708f82e408 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.037673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4871d-130d-4efd-8a4f-37708f82e408 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.038078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4871d-130d-4efd-8a4f-37708f82e408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.040617 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d9a4871d-130d-4efd-8a4f-37708f82e408 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:36.053232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791789ab-e556-41cd-9213-ae26ba4d4b49 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:36.058077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:36.058770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.059053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.059324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] PATH_INFO: `/v3/projects/1e82a86ff80d4fcb9c0ca76707f3d71b` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.060437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e82a86ff80d4fcb9c0ca76707f3d71b)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:36.062523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:36.108759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4871d-130d-4efd-8a4f-37708f82e408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:36.110267 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4871d-130d-4efd-8a4f-37708f82e408 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:36.119287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e82a86ff80d4fcb9c0ca76707f3d71b', 'request_id': 'req-b7a5fa92-1d72-45d6-877a-8072184c3bbf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:36.125143 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 797/1586] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:36 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:48:36.126469 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e82a86ff80d4fcb9c0ca76707f3d71b', 'request_id': 'req-b7a5fa92-1d72-45d6-877a-8072184c3bbf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:36.134095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e82a86ff80d4fcb9c0ca76707f3d71b', 'request_id': 'req-b7a5fa92-1d72-45d6-877a-8072184c3bbf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:36.153134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] The token cache is being invalidate because project 1e82a86ff80d4fcb9c0ca76707f3d71b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:36.153548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:36.154984 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b7a5fa92-1d72-45d6-877a-8072184c3bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:36.163594 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 790/1587] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:35 2022] DELETE /identity/v3/projects/1e82a86ff80d4fcb9c0ca76707f3d71b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:36.165555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ca15a-91a0-4321-8063-2873f9e13be2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.165861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ca15a-91a0-4321-8063-2873f9e13be2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.166140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ca15a-91a0-4321-8063-2873f9e13be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.200468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ee5ca15a-91a0-4321-8063-2873f9e13be2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:36.202864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ee5ca15a-91a0-4321-8063-2873f9e13be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:36.274492 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 798/1588] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:36.284680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d4009-73f5-435a-b1ed-2a1589499a37 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.287430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d4009-73f5-435a-b1ed-2a1589499a37 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.287430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d4009-73f5-435a-b1ed-2a1589499a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.317242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d94d4009-73f5-435a-b1ed-2a1589499a37 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:36.318497 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d94d4009-73f5-435a-b1ed-2a1589499a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:36.330316 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 791/1589] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:36.334817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ca15a-91a0-4321-8063-2873f9e13be2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:36.351371 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ca15a-91a0-4321-8063-2873f9e13be2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:36.360744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399d5d4-19b5-4b99-b5a3-1d9c7104c3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:36.361368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399d5d4-19b5-4b99-b5a3-1d9c7104c3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.361648 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399d5d4-19b5-4b99-b5a3-1d9c7104c3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.361944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399d5d4-19b5-4b99-b5a3-1d9c7104c3c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.362683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399d5d4-19b5-4b99-b5a3-1d9c7104c3c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:36.363269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399d5d4-19b5-4b99-b5a3-1d9c7104c3c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:36.364994 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399d5d4-19b5-4b99-b5a3-1d9c7104c3c7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:36.382755 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 799/1590] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:36.389351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d4009-73f5-435a-b1ed-2a1589499a37 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:36.402863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d4009-73f5-435a-b1ed-2a1589499a37 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:36.413665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7b150-963a-4d72-8bdd-ef3c9d4a3ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:36.414959 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7b150-963a-4d72-8bdd-ef3c9d4a3ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.415575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7b150-963a-4d72-8bdd-ef3c9d4a3ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.415895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7b150-963a-4d72-8bdd-ef3c9d4a3ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.416554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7b150-963a-4d72-8bdd-ef3c9d4a3ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:36.418631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7b150-963a-4d72-8bdd-ef3c9d4a3ed0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:36.463191 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 792/1591] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:36 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:36.467882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399d5d4-19b5-4b99-b5a3-1d9c7104c3c7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:36.480191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399d5d4-19b5-4b99-b5a3-1d9c7104c3c7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:36.543790 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:36.563285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae04a0-eb73-4224-a923-239d721eb8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:36.563285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae04a0-eb73-4224-a923-239d721eb8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.563285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae04a0-eb73-4224-a923-239d721eb8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.563285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae04a0-eb73-4224-a923-239d721eb8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.563615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae04a0-eb73-4224-a923-239d721eb8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:36.563615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae04a0-eb73-4224-a923-239d721eb8a0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:36.592010 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 800/1592] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:36 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:48:36.593349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7b150-963a-4d72-8bdd-ef3c9d4a3ed0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:36.614127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7b150-963a-4d72-8bdd-ef3c9d4a3ed0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:36.624094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ede2be-6c0f-4b58-bad6-0c2ba558b9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:36.624759 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ede2be-6c0f-4b58-bad6-0c2ba558b9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.625047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ede2be-6c0f-4b58-bad6-0c2ba558b9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.625315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ede2be-6c0f-4b58-bad6-0c2ba558b9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.626079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ede2be-6c0f-4b58-bad6-0c2ba558b9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:36.628443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ede2be-6c0f-4b58-bad6-0c2ba558b9d9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:36.642552 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 793/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:36.646647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae04a0-eb73-4224-a923-239d721eb8a0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:36.658682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae04a0-eb73-4224-a923-239d721eb8a0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:36.665642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5986ec-61f0-4136-b9dd-401141601290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:36.666538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5986ec-61f0-4136-b9dd-401141601290 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.667022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5986ec-61f0-4136-b9dd-401141601290 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.667515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5986ec-61f0-4136-b9dd-401141601290 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.668171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5986ec-61f0-4136-b9dd-401141601290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:36.669778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5986ec-61f0-4136-b9dd-401141601290 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:36.684089 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 801/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:36.688559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ede2be-6c0f-4b58-bad6-0c2ba558b9d9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:36.702532 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ede2be-6c0f-4b58-bad6-0c2ba558b9d9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:36.707493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87e81-ac34-430e-9e0d-075314f5bfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:36.708169 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87e81-ac34-430e-9e0d-075314f5bfd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.708564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87e81-ac34-430e-9e0d-075314f5bfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.708733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87e81-ac34-430e-9e0d-075314f5bfd8 admin admin] PATH_INFO: `/v3/projects/a2a05c44bf7745899305e49274c1b5ce/users/5a90736dc1024ff6be71b43988e903b3/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.726309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be87e81-ac34-430e-9e0d-075314f5bfd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a05c44bf7745899305e49274c1b5ce, user_id=5a90736dc1024ff6be71b43988e903b3, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:36.729908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be87e81-ac34-430e-9e0d-075314f5bfd8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:36.745841 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 794/1595] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:36 2022] PUT /identity/v3/projects/a2a05c44bf7745899305e49274c1b5ce/users/5a90736dc1024ff6be71b43988e903b3/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:36.758570 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe74fb-1782-47e3-b2d4-2f0244542919 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.758910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe74fb-1782-47e3-b2d4-2f0244542919 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.759378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe74fb-1782-47e3-b2d4-2f0244542919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.787795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4bfe74fb-1782-47e3-b2d4-2f0244542919 None None] MFA Rules not processed for user `5a90736dc1024ff6be71b43988e903b3`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:36.789008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4bfe74fb-1782-47e3-b2d4-2f0244542919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:36.866014 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 802/1596] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:36.873865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87e81-ac34-430e-9e0d-075314f5bfd8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:36.876126 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6be87e81-ac34-430e-9e0d-075314f5bfd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:36.947812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87e81-ac34-430e-9e0d-075314f5bfd8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:36.952642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38f98b-27c6-4dd1-bce5-5094366b4ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:36.953346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38f98b-27c6-4dd1-bce5-5094366b4ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:36.953630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38f98b-27c6-4dd1-bce5-5094366b4ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:36.953969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38f98b-27c6-4dd1-bce5-5094366b4ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:36.963210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38f98b-27c6-4dd1-bce5-5094366b4ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:36.964635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38f98b-27c6-4dd1-bce5-5094366b4ec2 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:36.974313 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 795/1597] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:39.627936 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe74fb-1782-47e3-b2d4-2f0244542919 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:39.695189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe74fb-1782-47e3-b2d4-2f0244542919 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:39.700500 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:39.701142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:39.701398 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:39.701665 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] PATH_INFO: `/v3/users/5a90736dc1024ff6be71b43988e903b3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:39.702816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a90736dc1024ff6be71b43988e903b3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:39.704796 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:39.753313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a90736dc1024ff6be71b43988e903b3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:39.759865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a90736dc1024ff6be71b43988e903b3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:39.760527 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a90736dc1024ff6be71b43988e903b3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:39.768702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a90736dc1024ff6be71b43988e903b3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:39.769793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:39.782752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a90736dc1024ff6be71b43988e903b3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:39.788585 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 803/1598] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:39 2022] DELETE /identity/v3/users/5a90736dc1024ff6be71b43988e903b3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:39.793742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd8b1e-0d23-41b5-8287-5d88da20ddb5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:39.794201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd8b1e-0d23-41b5-8287-5d88da20ddb5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:39.794414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd8b1e-0d23-41b5-8287-5d88da20ddb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:39.807609 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:39.814616 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-eafd8b1e-0d23-41b5-8287-5d88da20ddb5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:39.816804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-eafd8b1e-0d23-41b5-8287-5d88da20ddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:39.879014 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 796/1599] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:39.888813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:39.890192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:39.945995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33652de9-6447-45c5-8230-3af8fe5229ea admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:39.949605 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cbe30-969d-4c06-9a58-ac7b4fc0591d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:39.950227 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cbe30-969d-4c06-9a58-ac7b4fc0591d service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:39.950505 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cbe30-969d-4c06-9a58-ac7b4fc0591d service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:39.950824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cbe30-969d-4c06-9a58-ac7b4fc0591d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:39.959885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cbe30-969d-4c06-9a58-ac7b4fc0591d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:39.961321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cbe30-969d-4c06-9a58-ac7b4fc0591d service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:39.970558 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 804/1600] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:40.132572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd8b1e-0d23-41b5-8287-5d88da20ddb5 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:40.134074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-eafd8b1e-0d23-41b5-8287-5d88da20ddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.146230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd8b1e-0d23-41b5-8287-5d88da20ddb5 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:40.151074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:40.151697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.151971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.152238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] PATH_INFO: `/v3/users/93918ababe02454d9c4f4767d98a4897` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.153368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93918ababe02454d9c4f4767d98a4897)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:40.155296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:40.212373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93918ababe02454d9c4f4767d98a4897'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.213083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93918ababe02454d9c4f4767d98a4897'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.223171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93918ababe02454d9c4f4767d98a4897'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.233224 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93918ababe02454d9c4f4767d98a4897'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.245161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cbe30-969d-4c06-9a58-ac7b4fc0591d service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:40.246578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b85cbe30-969d-4c06-9a58-ac7b4fc0591d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.247423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-026203ea-9716-46a4-a034-6e558bb70c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93918ababe02454d9c4f4767d98a4897'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.261043 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 797/1601] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:40 2022] DELETE /identity/v3/users/93918ababe02454d9c4f4767d98a4897 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:40.265465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cbe30-969d-4c06-9a58-ac7b4fc0591d service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:40.265972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f05a3-fef4-4132-a755-84ea4c2b886a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.267622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f05a3-fef4-4132-a755-84ea4c2b886a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.268020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f05a3-fef4-4132-a755-84ea4c2b886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.298965 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c05f05a3-fef4-4132-a755-84ea4c2b886a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:40.302857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c05f05a3-fef4-4132-a755-84ea4c2b886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.343261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:40.344238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.344656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.345389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] PATH_INFO: `/v3/projects/a2a05c44bf7745899305e49274c1b5ce` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.347117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a05c44bf7745899305e49274c1b5ce)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:40.349509 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:40.366462 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 798/1602] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:40.381929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f05a3-fef4-4132-a755-84ea4c2b886a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:40.383679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c05f05a3-fef4-4132-a755-84ea4c2b886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.397067 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f05a3-fef4-4132-a755-84ea4c2b886a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:40.407530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a05c44bf7745899305e49274c1b5ce', 'request_id': 'req-c300c44d-d315-4b2f-a510-aa81f039993d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.414039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a05c44bf7745899305e49274c1b5ce', 'request_id': 'req-c300c44d-d315-4b2f-a510-aa81f039993d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.416071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:40.422158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a05c44bf7745899305e49274c1b5ce', 'request_id': 'req-c300c44d-d315-4b2f-a510-aa81f039993d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.438836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] The token cache is being invalidate because project a2a05c44bf7745899305e49274c1b5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:40.439219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.439863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.441695 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 805/1603] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:40 2022] DELETE /identity/v3/projects/a2a05c44bf7745899305e49274c1b5ce => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:40.442595 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:40.444082 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.459209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:40.461149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:40.461879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.462966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.462966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] PATH_INFO: `/v3/users/71b24019a949461a8863db9dd70d93b2` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.463700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] RBAC: Authorizing `identity:delete_user(user_id=71b24019a949461a8863db9dd70d93b2)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:40.465684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:40.513261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300c44d-d315-4b2f-a510-aa81f039993d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:40.517469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a610a-4ab7-4f46-836d-e36764188e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:40.518178 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a610a-4ab7-4f46-836d-e36764188e0f service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.518461 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a610a-4ab7-4f46-836d-e36764188e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.518758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a610a-4ab7-4f46-836d-e36764188e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.520641 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c28a610a-4ab7-4f46-836d-e36764188e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.523828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71b24019a949461a8863db9dd70d93b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.524553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71b24019a949461a8863db9dd70d93b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.533021 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71b24019a949461a8863db9dd70d93b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.536175 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a610a-4ab7-4f46-836d-e36764188e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:40.538192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a610a-4ab7-4f46-836d-e36764188e0f service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:40.543194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71b24019a949461a8863db9dd70d93b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.549119 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 806/1604] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:40.550385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b57c4-0041-4f89-b588-9f732dff00ff None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.550657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b57c4-0041-4f89-b588-9f732dff00ff None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.550962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b57c4-0041-4f89-b588-9f732dff00ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.565299 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dab12e15-cce9-47c0-b416-553c3e4c091d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71b24019a949461a8863db9dd70d93b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:40.574848 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 799/1605] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:40 2022] DELETE /identity/v3/users/71b24019a949461a8863db9dd70d93b2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:40.577521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-987b57c4-0041-4f89-b588-9f732dff00ff None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:40.578735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-987b57c4-0041-4f89-b588-9f732dff00ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.580137 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da855ddd-bb6e-49d2-b9cd-00ada772846f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.580430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da855ddd-bb6e-49d2-b9cd-00ada772846f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.580704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da855ddd-bb6e-49d2-b9cd-00ada772846f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.599909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-da855ddd-bb6e-49d2-b9cd-00ada772846f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:40.601162 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-da855ddd-bb6e-49d2-b9cd-00ada772846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.656125 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 807/1606] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:40.664010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151a5a7-21ae-4a9f-b849-936b3dbabf78 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.664526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151a5a7-21ae-4a9f-b849-936b3dbabf78 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.665038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151a5a7-21ae-4a9f-b849-936b3dbabf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.688850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8151a5a7-21ae-4a9f-b849-936b3dbabf78 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:40.690152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8151a5a7-21ae-4a9f-b849-936b3dbabf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.698057 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 800/1607] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:40.704731 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da855ddd-bb6e-49d2-b9cd-00ada772846f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:40.705887 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 808/1608] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:40.706104 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-da855ddd-bb6e-49d2-b9cd-00ada772846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:40.707080 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151a5a7-21ae-4a9f-b849-936b3dbabf78 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:40.718059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151a5a7-21ae-4a9f-b849-936b3dbabf78 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:40.723048 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c19040-c977-44da-99f6-0ba41197317d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:40.723710 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c19040-c977-44da-99f6-0ba41197317d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.724046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c19040-c977-44da-99f6-0ba41197317d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.724323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c19040-c977-44da-99f6-0ba41197317d admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.724986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c19040-c977-44da-99f6-0ba41197317d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:40.725423 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c19040-c977-44da-99f6-0ba41197317d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:40.727460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c19040-c977-44da-99f6-0ba41197317d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:40.740871 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 809/1609] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:40.745463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c19040-c977-44da-99f6-0ba41197317d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:40.755524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c19040-c977-44da-99f6-0ba41197317d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:40.757712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:40.760713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309325d-79e6-41d8-8b39-a51add1c9188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:40.762002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309325d-79e6-41d8-8b39-a51add1c9188 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.762893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309325d-79e6-41d8-8b39-a51add1c9188 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.763620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309325d-79e6-41d8-8b39-a51add1c9188 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.764891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309325d-79e6-41d8-8b39-a51add1c9188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:40.767290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309325d-79e6-41d8-8b39-a51add1c9188 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:40.774538 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da855ddd-bb6e-49d2-b9cd-00ada772846f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:40.794100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa6eb4-3378-4fe9-b4c5-321c912e2db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:40.795827 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa6eb4-3378-4fe9-b4c5-321c912e2db5 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.796281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa6eb4-3378-4fe9-b4c5-321c912e2db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.796695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa6eb4-3378-4fe9-b4c5-321c912e2db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.810210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa6eb4-3378-4fe9-b4c5-321c912e2db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:40.811787 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa6eb4-3378-4fe9-b4c5-321c912e2db5 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:40.817940 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 810/1610] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:40 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:48:40.819193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309325d-79e6-41d8-8b39-a51add1c9188 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:40.823616 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 801/1611] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:48:40.944890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309325d-79e6-41d8-8b39-a51add1c9188 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:40.958247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b2787-c0a1-4bd3-a485-ed1b2c302fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:40.959157 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2787-c0a1-4bd3-a485-ed1b2c302fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:40.959157 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2787-c0a1-4bd3-a485-ed1b2c302fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:40.959448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2787-c0a1-4bd3-a485-ed1b2c302fb9 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:40.960091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b2787-c0a1-4bd3-a485-ed1b2c302fb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:40.964156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b2787-c0a1-4bd3-a485-ed1b2c302fb9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:41.021623 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 811/1612] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:40 2022] POST /identity/v3/users => generated 285 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:41.027628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa6eb4-3378-4fe9-b4c5-321c912e2db5 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:41.043136 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa6eb4-3378-4fe9-b4c5-321c912e2db5 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:41.050028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c80906-f7fe-4c17-9b24-f275b11f2e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:41.050864 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c80906-f7fe-4c17-9b24-f275b11f2e8e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.051212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c80906-f7fe-4c17-9b24-f275b11f2e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.051615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c80906-f7fe-4c17-9b24-f275b11f2e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.052612 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c80906-f7fe-4c17-9b24-f275b11f2e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:41.054815 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c80906-f7fe-4c17-9b24-f275b11f2e8e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:41.060627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b2787-c0a1-4bd3-a485-ed1b2c302fb9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:41.062650 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-188b2787-c0a1-4bd3-a485-ed1b2c302fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.077030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:41.093268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b2787-c0a1-4bd3-a485-ed1b2c302fb9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:41.102001 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 802/1613] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:41.104463 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c80906-f7fe-4c17-9b24-f275b11f2e8e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:41.115059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c80906-f7fe-4c17-9b24-f275b11f2e8e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:41.116492 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:41.120232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d0e0-1e28-4cf6-ae8f-7fe30324c083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:41.120927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d0e0-1e28-4cf6-ae8f-7fe30324c083 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.121278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d0e0-1e28-4cf6-ae8f-7fe30324c083 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.121515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.121737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d0e0-1e28-4cf6-ae8f-7fe30324c083 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.122492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d0e0-1e28-4cf6-ae8f-7fe30324c083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:41.123414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.123733 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] PATH_INFO: `/v3/projects/0498a8458fba445198f5a4f3a706fd88` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.124699 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d0e0-1e28-4cf6-ae8f-7fe30324c083 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:41.129839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0498a8458fba445198f5a4f3a706fd88)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:41.141527 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 803/1614] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:41.145964 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d0e0-1e28-4cf6-ae8f-7fe30324c083 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:41.147302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:41.164494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d0e0-1e28-4cf6-ae8f-7fe30324c083 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:41.170514 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4445b5f-8341-4779-9ed8-378b1d7b0c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:41.171686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4445b5f-8341-4779-9ed8-378b1d7b0c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.172092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4445b5f-8341-4779-9ed8-378b1d7b0c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.172702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4445b5f-8341-4779-9ed8-378b1d7b0c3e admin admin] PATH_INFO: `/v3/projects/cd8f88e9f153403a8e1e93dbc6c8b181/users/e32015b8cacc442eb3533ba2b290da4b/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.191032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4445b5f-8341-4779-9ed8-378b1d7b0c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8f88e9f153403a8e1e93dbc6c8b181, user_id=e32015b8cacc442eb3533ba2b290da4b, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:41.194245 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4445b5f-8341-4779-9ed8-378b1d7b0c3e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:41.235356 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 804/1615] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:41 2022] PUT /identity/v3/projects/cd8f88e9f153403a8e1e93dbc6c8b181/users/e32015b8cacc442eb3533ba2b290da4b/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:41.246563 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ab5cf-a51c-4506-b745-5a4013abbd3b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.247101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ab5cf-a51c-4506-b745-5a4013abbd3b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.247696 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ab5cf-a51c-4506-b745-5a4013abbd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.272506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0498a8458fba445198f5a4f3a706fd88', 'request_id': 'req-b3861221-1c7f-4a02-b5ad-d30fc85d5381'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.295696 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0498a8458fba445198f5a4f3a706fd88', 'request_id': 'req-b3861221-1c7f-4a02-b5ad-d30fc85d5381'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.313405 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0498a8458fba445198f5a4f3a706fd88', 'request_id': 'req-b3861221-1c7f-4a02-b5ad-d30fc85d5381'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.323723 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-214ab5cf-a51c-4506-b745-5a4013abbd3b None None] MFA Rules not processed for user `e32015b8cacc442eb3533ba2b290da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:41.325773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-214ab5cf-a51c-4506-b745-5a4013abbd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.337967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] The token cache is being invalidate because project 0498a8458fba445198f5a4f3a706fd88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:41.338268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.339087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b3861221-1c7f-4a02-b5ad-d30fc85d5381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.341676 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 812/1616] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:41 2022] DELETE /identity/v3/projects/0498a8458fba445198f5a4f3a706fd88 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:41.349442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81bdc5-f623-4e2f-b6d7-6ff36e65bb84 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.350156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81bdc5-f623-4e2f-b6d7-6ff36e65bb84 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.350814 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81bdc5-f623-4e2f-b6d7-6ff36e65bb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.400518 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bb81bdc5-f623-4e2f-b6d7-6ff36e65bb84 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:41.401759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bb81bdc5-f623-4e2f-b6d7-6ff36e65bb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.426374 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 805/1617] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:41.427356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ab5cf-a51c-4506-b745-5a4013abbd3b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:41.431510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-214ab5cf-a51c-4506-b745-5a4013abbd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.466379 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 813/1618] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:48:41.467125 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81bdc5-f623-4e2f-b6d7-6ff36e65bb84 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:41.468722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bb81bdc5-f623-4e2f-b6d7-6ff36e65bb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.510203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ab5cf-a51c-4506-b745-5a4013abbd3b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:41.515256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:41.515892 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.516171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.516445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] PATH_INFO: `/v3/projects/31049e0e2ef84586930562d555f1bcc3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.517555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31049e0e2ef84586930562d555f1bcc3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:41.519552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:41.524267 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81bdc5-f623-4e2f-b6d7-6ff36e65bb84 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:41.527902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167077ac-93e3-4baa-9bb6-fc056ad2cd14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:41.528673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167077ac-93e3-4baa-9bb6-fc056ad2cd14 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.528907 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167077ac-93e3-4baa-9bb6-fc056ad2cd14 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.529253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167077ac-93e3-4baa-9bb6-fc056ad2cd14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.537458 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167077ac-93e3-4baa-9bb6-fc056ad2cd14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:41.538970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167077ac-93e3-4baa-9bb6-fc056ad2cd14 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:41.548497 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 814/1619] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:41.549684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79674c9-bd49-466b-b8a8-e6eed92e283a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.549966 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79674c9-bd49-466b-b8a8-e6eed92e283a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.550242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79674c9-bd49-466b-b8a8-e6eed92e283a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.552194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:41.571476 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d79674c9-bd49-466b-b8a8-e6eed92e283a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:41.572970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d79674c9-bd49-466b-b8a8-e6eed92e283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.580343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31049e0e2ef84586930562d555f1bcc3', 'request_id': 'req-6fc306b3-7194-4c8c-970c-59e6efcafce7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.587825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31049e0e2ef84586930562d555f1bcc3', 'request_id': 'req-6fc306b3-7194-4c8c-970c-59e6efcafce7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.595237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31049e0e2ef84586930562d555f1bcc3', 'request_id': 'req-6fc306b3-7194-4c8c-970c-59e6efcafce7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.600578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:41.612544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] The token cache is being invalidate because project 31049e0e2ef84586930562d555f1bcc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:41.612745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.613552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.615338 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 806/1620] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:41 2022] DELETE /identity/v3/projects/31049e0e2ef84586930562d555f1bcc3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:41.616226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:41.617546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.643059 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 815/1621] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:41.643970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79674c9-bd49-466b-b8a8-e6eed92e283a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:41.647562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d79674c9-bd49-466b-b8a8-e6eed92e283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.660121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79674c9-bd49-466b-b8a8-e6eed92e283a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:41.664949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:41.665940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.666213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.666480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] PATH_INFO: `/v3/users/e989127ad9514a24bb389ee3a5a9e63e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.667735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e989127ad9514a24bb389ee3a5a9e63e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:41.670317 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:41.712223 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc306b3-7194-4c8c-970c-59e6efcafce7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:41.719432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:41.720086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.720374 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.720665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] PATH_INFO: `/v3/users/e32015b8cacc442eb3533ba2b290da4b` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.722063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32015b8cacc442eb3533ba2b290da4b)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:41.724168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:41.763102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e989127ad9514a24bb389ee3a5a9e63e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.774153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e989127ad9514a24bb389ee3a5a9e63e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.774978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e989127ad9514a24bb389ee3a5a9e63e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.783877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e989127ad9514a24bb389ee3a5a9e63e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.787961 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32015b8cacc442eb3533ba2b290da4b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.788898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32015b8cacc442eb3533ba2b290da4b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.797804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32015b8cacc442eb3533ba2b290da4b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.802579 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e989127ad9514a24bb389ee3a5a9e63e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.807105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32015b8cacc442eb3533ba2b290da4b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.810342 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 816/1622] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:41 2022] DELETE /identity/v3/users/e989127ad9514a24bb389ee3a5a9e63e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:41.811264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:41.812624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.844073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f14dd38f-2643-461c-bf8c-c9182ae71d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32015b8cacc442eb3533ba2b290da4b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:41.855884 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 807/1623] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:41 2022] DELETE /identity/v3/users/e32015b8cacc442eb3533ba2b290da4b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:41.858125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0412e0-2283-4026-a212-8822afc6edcd None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:41.858481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0412e0-2283-4026-a212-8822afc6edcd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:41.858879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0412e0-2283-4026-a212-8822afc6edcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:41.895829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5a0412e0-2283-4026-a212-8822afc6edcd None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:41.897169 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5a0412e0-2283-4026-a212-8822afc6edcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:41.918390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bab02-5148-4b91-819e-edc2402ebe2f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:41.974950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:41.989206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:42.022143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.024771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.026652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.027110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.027840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:42.028340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.031862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.056263 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 817/1624] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:42.059045 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.061816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.082937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bbd15-c025-4ce3-b8bb-601e3aa7b3ac admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.089698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25200c2-6737-4b77-89c1-cd97c4273309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.091224 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 808/1625] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:42.094128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25200c2-6737-4b77-89c1-cd97c4273309 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.094755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25200c2-6737-4b77-89c1-cd97c4273309 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.095187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25200c2-6737-4b77-89c1-cd97c4273309 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.096059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25200c2-6737-4b77-89c1-cd97c4273309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.098190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25200c2-6737-4b77-89c1-cd97c4273309 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.109362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0412e0-2283-4026-a212-8822afc6edcd None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.111131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5a0412e0-2283-4026-a212-8822afc6edcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.149643 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 818/1626] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:48:42.150893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25200c2-6737-4b77-89c1-cd97c4273309 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.195679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0412e0-2283-4026-a212-8822afc6edcd None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.245076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25200c2-6737-4b77-89c1-cd97c4273309 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.252182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fe944-ae47-41ff-9d39-aa90fd904c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.252822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fe944-ae47-41ff-9d39-aa90fd904c38 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.253501 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fe944-ae47-41ff-9d39-aa90fd904c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.253501 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fe944-ae47-41ff-9d39-aa90fd904c38 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.254254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fe944-ae47-41ff-9d39-aa90fd904c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.256311 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fe944-ae47-41ff-9d39-aa90fd904c38 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.262563 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610040a-5842-4bf2-aa15-43ce7ce1b39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.265458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610040a-5842-4bf2-aa15-43ce7ce1b39d service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.265458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610040a-5842-4bf2-aa15-43ce7ce1b39d service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.265458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610040a-5842-4bf2-aa15-43ce7ce1b39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.280441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610040a-5842-4bf2-aa15-43ce7ce1b39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.282511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610040a-5842-4bf2-aa15-43ce7ce1b39d service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.290135 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 819/1627] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:42.294877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fe944-ae47-41ff-9d39-aa90fd904c38 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.296725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8d3fe944-ae47-41ff-9d39-aa90fd904c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.297260 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 809/1628] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:48:42.298197 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610040a-5842-4bf2-aa15-43ce7ce1b39d service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.308096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610040a-5842-4bf2-aa15-43ce7ce1b39d service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.313248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8115449-3018-42cf-9fd0-8b47ce51057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.313870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8115449-3018-42cf-9fd0-8b47ce51057a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.314144 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8115449-3018-42cf-9fd0-8b47ce51057a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.314410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8115449-3018-42cf-9fd0-8b47ce51057a admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.315097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8115449-3018-42cf-9fd0-8b47ce51057a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.316696 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8115449-3018-42cf-9fd0-8b47ce51057a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.319520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fe944-ae47-41ff-9d39-aa90fd904c38 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.326845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.327651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.328028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.328256 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 810/1629] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:42.328785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] PATH_INFO: `/v3/projects/93e1ad5db0304facbcf9242bf687f347` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.330072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e1ad5db0304facbcf9242bf687f347)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.332187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.332297 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8115449-3018-42cf-9fd0-8b47ce51057a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.342577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8115449-3018-42cf-9fd0-8b47ce51057a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.348322 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb37a0-d7ff-475b-9f3b-c643adc13f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.349040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb37a0-d7ff-475b-9f3b-c643adc13f1c admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.349434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb37a0-d7ff-475b-9f3b-c643adc13f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.349715 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb37a0-d7ff-475b-9f3b-c643adc13f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.350352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb37a0-d7ff-475b-9f3b-c643adc13f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.351995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb37a0-d7ff-475b-9f3b-c643adc13f1c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.367575 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 811/1630] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:48:42.372053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb37a0-d7ff-475b-9f3b-c643adc13f1c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.384622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb37a0-d7ff-475b-9f3b-c643adc13f1c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.393134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e1ad5db0304facbcf9242bf687f347', 'request_id': 'req-4952b659-e417-4b22-bef8-801b4249cf95'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.402597 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e1ad5db0304facbcf9242bf687f347', 'request_id': 'req-4952b659-e417-4b22-bef8-801b4249cf95'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.413891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e1ad5db0304facbcf9242bf687f347', 'request_id': 'req-4952b659-e417-4b22-bef8-801b4249cf95'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.433607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] The token cache is being invalidate because project 93e1ad5db0304facbcf9242bf687f347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:42.434051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.434901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4952b659-e417-4b22-bef8-801b4249cf95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.437528 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 820/1631] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:42 2022] DELETE /identity/v3/projects/93e1ad5db0304facbcf9242bf687f347 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:42.442877 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6c33c-8e03-42f4-8dd8-c1bc095b698b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.443352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6c33c-8e03-42f4-8dd8-c1bc095b698b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.443722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc6c33c-8e03-42f4-8dd8-c1bc095b698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.459725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897d3fd-3cd8-4947-b0b8-e20d5753ed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.460665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d3fd-3cd8-4947-b0b8-e20d5753ed0a admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.461575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d3fd-3cd8-4947-b0b8-e20d5753ed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.461990 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897d3fd-3cd8-4947-b0b8-e20d5753ed0a admin admin] PATH_INFO: `/v3/projects/d8cb97e6993049e0840e291e83f3a55f/users/6251b7909a7b43a0bec0de0392aee1e0/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.470636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9cc6c33c-8e03-42f4-8dd8-c1bc095b698b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:42.472150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9cc6c33c-8e03-42f4-8dd8-c1bc095b698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.487637 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 821/1632] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:42.489266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d795a-3201-49a5-b296-e39abd18f01a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.489554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d795a-3201-49a5-b296-e39abd18f01a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.489906 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d795a-3201-49a5-b296-e39abd18f01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.492298 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d3fd-3cd8-4947-b0b8-e20d5753ed0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cb97e6993049e0840e291e83f3a55f, user_id=6251b7909a7b43a0bec0de0392aee1e0, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.494995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897d3fd-3cd8-4947-b0b8-e20d5753ed0a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.501588 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:42.526769 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 812/1633] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:42 2022] PUT /identity/v3/projects/d8cb97e6993049e0840e291e83f3a55f/users/6251b7909a7b43a0bec0de0392aee1e0/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:42.527221 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2f3d795a-3201-49a5-b296-e39abd18f01a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:42.528676 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2f3d795a-3201-49a5-b296-e39abd18f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.536775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bae63-c501-4e07-b96d-61cf8f8f9a3b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.537281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bae63-c501-4e07-b96d-61cf8f8f9a3b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.537696 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bae63-c501-4e07-b96d-61cf8f8f9a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.569861 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9c7bae63-c501-4e07-b96d-61cf8f8f9a3b None None] MFA Rules not processed for user `6251b7909a7b43a0bec0de0392aee1e0`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:42.571453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9c7bae63-c501-4e07-b96d-61cf8f8f9a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.581731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:42.649663 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 822/1634] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:42.649663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d795a-3201-49a5-b296-e39abd18f01a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.649663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2f3d795a-3201-49a5-b296-e39abd18f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.660375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d795a-3201-49a5-b296-e39abd18f01a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.663609 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 813/1635] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:42.664411 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bae63-c501-4e07-b96d-61cf8f8f9a3b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.665535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.666184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.666456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.666766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] PATH_INFO: `/v3/projects/cd8f88e9f153403a8e1e93dbc6c8b181` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.668228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8f88e9f153403a8e1e93dbc6c8b181)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.671781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.676427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bae63-c501-4e07-b96d-61cf8f8f9a3b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.681450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.682164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.682506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.682879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.683630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:42.683934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.685522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.697371 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 814/1636] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:42.698072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.699678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.712666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8726d-6bfb-4686-b132-02c90bf14bed admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.724818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8f88e9f153403a8e1e93dbc6c8b181', 'request_id': 'req-624bb948-4797-46d7-b56e-c988031fffe1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.730847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8f88e9f153403a8e1e93dbc6c8b181', 'request_id': 'req-624bb948-4797-46d7-b56e-c988031fffe1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.737850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8f88e9f153403a8e1e93dbc6c8b181', 'request_id': 'req-624bb948-4797-46d7-b56e-c988031fffe1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.750929 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] The token cache is being invalidate because project cd8f88e9f153403a8e1e93dbc6c8b181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:42.751225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.751851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:42.753367 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 823/1637] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:42 2022] DELETE /identity/v3/projects/cd8f88e9f153403a8e1e93dbc6c8b181 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:42.754510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.755629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.772580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc2c9e-5aff-4f63-b95b-c079e1516c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.773219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc2c9e-5aff-4f63-b95b-c079e1516c2a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.773496 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc2c9e-5aff-4f63-b95b-c079e1516c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.773779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc2c9e-5aff-4f63-b95b-c079e1516c2a admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.774433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc2c9e-5aff-4f63-b95b-c079e1516c2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.776388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc2c9e-5aff-4f63-b95b-c079e1516c2a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.805425 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 815/1638] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:42.806677 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b686a84-e870-4ea0-a5c5-56ecd2572d6d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.806985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b686a84-e870-4ea0-a5c5-56ecd2572d6d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.807244 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b686a84-e870-4ea0-a5c5-56ecd2572d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.822167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bb948-4797-46d7-b56e-c988031fffe1 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.825643 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9b686a84-e870-4ea0-a5c5-56ecd2572d6d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:42.826877 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9b686a84-e870-4ea0-a5c5-56ecd2572d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.879364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:42.883135 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14970588-368a-4876-a202-3f33bf4741b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.883761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14970588-368a-4876-a202-3f33bf4741b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.884176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14970588-368a-4876-a202-3f33bf4741b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.884453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14970588-368a-4876-a202-3f33bf4741b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.885091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14970588-368a-4876-a202-3f33bf4741b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.887084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14970588-368a-4876-a202-3f33bf4741b5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.897708 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 816/1639] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:42.897876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:42.897998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b686a84-e870-4ea0-a5c5-56ecd2572d6d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.903116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9b686a84-e870-4ea0-a5c5-56ecd2572d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.914935 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b686a84-e870-4ea0-a5c5-56ecd2572d6d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:42.920875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d0ece-588c-4bb2-b87f-1c3f80073023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:42.921397 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 824/1640] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:42.921525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0ece-588c-4bb2-b87f-1c3f80073023 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.921805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0ece-588c-4bb2-b87f-1c3f80073023 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.922044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d0ece-588c-4bb2-b87f-1c3f80073023 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.922666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b894d-e432-41b9-a6b3-f1c7e7825967 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:42.922801 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d0ece-588c-4bb2-b87f-1c3f80073023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:42.923014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b894d-e432-41b9-a6b3-f1c7e7825967 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:42.923264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b894d-e432-41b9-a6b3-f1c7e7825967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:42.924673 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d0ece-588c-4bb2-b87f-1c3f80073023 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:42.947978 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 817/1641] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/users => generated 293 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:42.948784 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d0ece-588c-4bb2-b87f-1c3f80073023 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:42.949531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3d3b894d-e432-41b9-a6b3-f1c7e7825967 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:42.950831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3d3b894d-e432-41b9-a6b3-f1c7e7825967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:42.958962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d0ece-588c-4bb2-b87f-1c3f80073023 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.036380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352a3f-2516-442c-b3d3-dbcef0c09f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.037278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352a3f-2516-442c-b3d3-dbcef0c09f2f admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.037627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352a3f-2516-442c-b3d3-dbcef0c09f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.037993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7352a3f-2516-442c-b3d3-dbcef0c09f2f admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.038780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7352a3f-2516-442c-b3d3-dbcef0c09f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.041218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7352a3f-2516-442c-b3d3-dbcef0c09f2f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.067772 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 818/1642] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:43.067772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352a3f-2516-442c-b3d3-dbcef0c09f2f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.076571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7352a3f-2516-442c-b3d3-dbcef0c09f2f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.081246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153e71e-edcc-40a0-8443-c4c7668d0a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.081878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153e71e-edcc-40a0-8443-c4c7668d0a1a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.082156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153e71e-edcc-40a0-8443-c4c7668d0a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.082429 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153e71e-edcc-40a0-8443-c4c7668d0a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.083180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153e71e-edcc-40a0-8443-c4c7668d0a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.084801 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153e71e-edcc-40a0-8443-c4c7668d0a1a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.096780 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 819/1643] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:43.097718 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153e71e-edcc-40a0-8443-c4c7668d0a1a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.108287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153e71e-edcc-40a0-8443-c4c7668d0a1a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.113242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2d236-c05a-48a7-ba89-192b0145fb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.114025 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2d236-c05a-48a7-ba89-192b0145fb15 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.114377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2d236-c05a-48a7-ba89-192b0145fb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.114860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2d236-c05a-48a7-ba89-192b0145fb15 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.115569 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2d236-c05a-48a7-ba89-192b0145fb15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.117222 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2d236-c05a-48a7-ba89-192b0145fb15 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.127669 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 820/1644] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:43.128275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2d236-c05a-48a7-ba89-192b0145fb15 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.137686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2d236-c05a-48a7-ba89-192b0145fb15 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.142182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9dd9-0cb2-46b7-923b-acf8b6b02874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.142786 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9dd9-0cb2-46b7-923b-acf8b6b02874 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.143065 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9dd9-0cb2-46b7-923b-acf8b6b02874 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.143432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9dd9-0cb2-46b7-923b-acf8b6b02874 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.143524 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 825/1645] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:43.143943 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d9dd9-0cb2-46b7-923b-acf8b6b02874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.144466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b894d-e432-41b9-a6b3-f1c7e7825967 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.145612 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d9dd9-0cb2-46b7-923b-acf8b6b02874 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.155620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b894d-e432-41b9-a6b3-f1c7e7825967 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.156091 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 821/1646] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:43.159909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9dd9-0cb2-46b7-923b-acf8b6b02874 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.160381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272afd50-ec67-4f01-8dbf-d0f42a6fe9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.161024 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272afd50-ec67-4f01-8dbf-d0f42a6fe9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.161301 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272afd50-ec67-4f01-8dbf-d0f42a6fe9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.161572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272afd50-ec67-4f01-8dbf-d0f42a6fe9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.162225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272afd50-ec67-4f01-8dbf-d0f42a6fe9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.163873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272afd50-ec67-4f01-8dbf-d0f42a6fe9b6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.169911 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9dd9-0cb2-46b7-923b-acf8b6b02874 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.175163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1a964-ddfb-4aef-8d45-cac1bcf66ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.175864 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 826/1647] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:43.175967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1a964-ddfb-4aef-8d45-cac1bcf66ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.175967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1a964-ddfb-4aef-8d45-cac1bcf66ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.176334 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1a964-ddfb-4aef-8d45-cac1bcf66ad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.176611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272afd50-ec67-4f01-8dbf-d0f42a6fe9b6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.177235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1a964-ddfb-4aef-8d45-cac1bcf66ad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:43.177537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1a964-ddfb-4aef-8d45-cac1bcf66ad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.179367 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1a964-ddfb-4aef-8d45-cac1bcf66ad7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.186560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272afd50-ec67-4f01-8dbf-d0f42a6fe9b6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.191706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7d460-e6e9-43fd-ac25-f139f6f5b5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.192811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7d460-e6e9-43fd-ac25-f139f6f5b5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.192811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7d460-e6e9-43fd-ac25-f139f6f5b5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.192970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7d460-e6e9-43fd-ac25-f139f6f5b5fe admin admin] PATH_INFO: `/v3/projects/659fad43bc3347fc962a899a4641b7da/users/43e7770fc3d64657b7a716fe5ac02f6c/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.194076 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 822/1648] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:43.195128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1a964-ddfb-4aef-8d45-cac1bcf66ad7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.206075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1a964-ddfb-4aef-8d45-cac1bcf66ad7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.211397 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd72bd-7255-4e5f-92cd-9c0a8ee56327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.212579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd72bd-7255-4e5f-92cd-9c0a8ee56327 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.212997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd72bd-7255-4e5f-92cd-9c0a8ee56327 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.213536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd72bd-7255-4e5f-92cd-9c0a8ee56327 admin admin] PATH_INFO: `/v3/projects/8955229bb756458fb8202cae89a62243/users/20284690f04a433188e0aefad57246e7/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.220793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7d460-e6e9-43fd-ac25-f139f6f5b5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=659fad43bc3347fc962a899a4641b7da, user_id=43e7770fc3d64657b7a716fe5ac02f6c, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.224029 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7d460-e6e9-43fd-ac25-f139f6f5b5fe admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.240349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd72bd-7255-4e5f-92cd-9c0a8ee56327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8955229bb756458fb8202cae89a62243, user_id=20284690f04a433188e0aefad57246e7, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.241535 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 827/1649] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:43 2022] PUT /identity/v3/projects/659fad43bc3347fc962a899a4641b7da/users/43e7770fc3d64657b7a716fe5ac02f6c/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:43.242625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7d460-e6e9-43fd-ac25-f139f6f5b5fe admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.244010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd72bd-7255-4e5f-92cd-9c0a8ee56327 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.263416 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 823/1650] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:43 2022] PUT /identity/v3/projects/8955229bb756458fb8202cae89a62243/users/20284690f04a433188e0aefad57246e7/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:43.264619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29028b-ee67-4cae-8aaf-61783f41b9dc None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.264905 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29028b-ee67-4cae-8aaf-61783f41b9dc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.265178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29028b-ee67-4cae-8aaf-61783f41b9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.316860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6c29028b-ee67-4cae-8aaf-61783f41b9dc None None] MFA Rules not processed for user `43e7770fc3d64657b7a716fe5ac02f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:43.318273 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6c29028b-ee67-4cae-8aaf-61783f41b9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:43.350069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7d460-e6e9-43fd-ac25-f139f6f5b5fe admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.370250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:43.390572 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 824/1651] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:43.392606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c7005-c0ab-41ed-ab65-99ac128aadcb None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.393125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c7005-c0ab-41ed-ab65-99ac128aadcb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.393589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c7005-c0ab-41ed-ab65-99ac128aadcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.439444 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bd1c7005-c0ab-41ed-ab65-99ac128aadcb None None] MFA Rules not processed for user `20284690f04a433188e0aefad57246e7`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:43.441401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:43.442070 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bd1c7005-c0ab-41ed-ab65-99ac128aadcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:43.452679 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab6399-99b5-4e54-b995-9b09ff5aaa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.453346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab6399-99b5-4e54-b995-9b09ff5aaa6c admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.453624 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab6399-99b5-4e54-b995-9b09ff5aaa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.453895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab6399-99b5-4e54-b995-9b09ff5aaa6c admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.454532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab6399-99b5-4e54-b995-9b09ff5aaa6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.456479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab6399-99b5-4e54-b995-9b09ff5aaa6c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.490457 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 828/1652] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:43 2022] POST /identity/v3/projects => generated 383 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:43.491940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab6399-99b5-4e54-b995-9b09ff5aaa6c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.517313 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 825/1653] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:43.518141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c7005-c0ab-41ed-ab65-99ac128aadcb None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.530011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c7005-c0ab-41ed-ab65-99ac128aadcb None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.578872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab6399-99b5-4e54-b995-9b09ff5aaa6c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.584889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec46aa7-4d8c-4f11-b345-1a6c8dd0900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.586079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec46aa7-4d8c-4f11-b345-1a6c8dd0900e admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.586443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec46aa7-4d8c-4f11-b345-1a6c8dd0900e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.586806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec46aa7-4d8c-4f11-b345-1a6c8dd0900e admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.587519 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec46aa7-4d8c-4f11-b345-1a6c8dd0900e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.589656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec46aa7-4d8c-4f11-b345-1a6c8dd0900e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.648922 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 829/1654] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:43 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:43.649966 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec46aa7-4d8c-4f11-b345-1a6c8dd0900e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.660728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b4ac4-81e1-48d1-ae44-ec83cd194211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.661024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b4ac4-81e1-48d1-ae44-ec83cd194211 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.661220 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b4ac4-81e1-48d1-ae44-ec83cd194211 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.664856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b4ac4-81e1-48d1-ae44-ec83cd194211 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.667557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b4ac4-81e1-48d1-ae44-ec83cd194211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.685082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b4ac4-81e1-48d1-ae44-ec83cd194211 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.735881 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 826/1655] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:43 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:43.736402 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b4ac4-81e1-48d1-ae44-ec83cd194211 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.747201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b4ac4-81e1-48d1-ae44-ec83cd194211 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.757057 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e957f-5f4c-48b7-8317-a1807c442e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.757782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e957f-5f4c-48b7-8317-a1807c442e83 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.758124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e957f-5f4c-48b7-8317-a1807c442e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.758458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e957f-5f4c-48b7-8317-a1807c442e83 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.759607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e957f-5f4c-48b7-8317-a1807c442e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.762019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e957f-5f4c-48b7-8317-a1807c442e83 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.764848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec46aa7-4d8c-4f11-b345-1a6c8dd0900e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.770390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ebab-aef2-46ce-b301-07b71c8e4675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.771297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ebab-aef2-46ce-b301-07b71c8e4675 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.771753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ebab-aef2-46ce-b301-07b71c8e4675 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.772216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ebab-aef2-46ce-b301-07b71c8e4675 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.773052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ebab-aef2-46ce-b301-07b71c8e4675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.775179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ebab-aef2-46ce-b301-07b71c8e4675 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.790497 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 827/1656] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:43 2022] POST /identity/v3/users => generated 298 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:43.791748 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e957f-5f4c-48b7-8317-a1807c442e83 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.804607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e957f-5f4c-48b7-8317-a1807c442e83 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.810144 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24373635-8077-4fd3-aa0e-fe1ccebde497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.810888 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24373635-8077-4fd3-aa0e-fe1ccebde497 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.811076 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24373635-8077-4fd3-aa0e-fe1ccebde497 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.811353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24373635-8077-4fd3-aa0e-fe1ccebde497 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.812008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24373635-8077-4fd3-aa0e-fe1ccebde497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.813610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24373635-8077-4fd3-aa0e-fe1ccebde497 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.816222 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 830/1657] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:43 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:43.817460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ebab-aef2-46ce-b301-07b71c8e4675 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.831076 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 828/1658] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:43.832247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24373635-8077-4fd3-aa0e-fe1ccebde497 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.842893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24373635-8077-4fd3-aa0e-fe1ccebde497 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.874711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:43.885810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ebab-aef2-46ce-b301-07b71c8e4675 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.891295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8f248-ad8c-49a4-a59c-dba79f199238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.892112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:43.892416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f248-ad8c-49a4-a59c-dba79f199238 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.892701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f248-ad8c-49a4-a59c-dba79f199238 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.892972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f248-ad8c-49a4-a59c-dba79f199238 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.893683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f248-ad8c-49a4-a59c-dba79f199238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.895390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f248-ad8c-49a4-a59c-dba79f199238 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.905735 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 831/1659] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:43.909033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8f248-ad8c-49a4-a59c-dba79f199238 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.915576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac469263-8dba-4574-a8b6-72264b29b584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.916189 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac469263-8dba-4574-a8b6-72264b29b584 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.916468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac469263-8dba-4574-a8b6-72264b29b584 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.916604 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8f248-ad8c-49a4-a59c-dba79f199238 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.916738 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac469263-8dba-4574-a8b6-72264b29b584 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.917373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac469263-8dba-4574-a8b6-72264b29b584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.919349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac469263-8dba-4574-a8b6-72264b29b584 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.921311 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd92f8-ea10-4079-9c10-cff292d4abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.921905 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd92f8-ea10-4079-9c10-cff292d4abd9 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.922183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd92f8-ea10-4079-9c10-cff292d4abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.922456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd92f8-ea10-4079-9c10-cff292d4abd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.923120 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd92f8-ea10-4079-9c10-cff292d4abd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.924720 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd92f8-ea10-4079-9c10-cff292d4abd9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.935889 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 832/1660] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:43.936757 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd92f8-ea10-4079-9c10-cff292d4abd9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.946376 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 829/1661] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:43 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:43.947511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac469263-8dba-4574-a8b6-72264b29b584 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.948133 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd92f8-ea10-4079-9c10-cff292d4abd9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.952601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42206d15-a4c4-4697-9230-2ab77fa8d1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.953813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42206d15-a4c4-4697-9230-2ab77fa8d1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.953813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42206d15-a4c4-4697-9230-2ab77fa8d1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.953813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42206d15-a4c4-4697-9230-2ab77fa8d1b7 admin admin] PATH_INFO: `/v3/projects/18c72efd99534605b177aa144ebb5f38/users/9bf9c97c7917458297b90c7c13524cc7/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.957024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac469263-8dba-4574-a8b6-72264b29b584 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:43.961719 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c983d9c-e895-4199-bcfa-8dba8cd0ba0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:43.962495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c983d9c-e895-4199-bcfa-8dba8cd0ba0b admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:43.962790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c983d9c-e895-4199-bcfa-8dba8cd0ba0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:43.963092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c983d9c-e895-4199-bcfa-8dba8cd0ba0b admin admin] PATH_INFO: `/v3/projects/238b1450574c411682599a39ad0a8371/users/1aa023d6ce6b45efa086c40c292f4ad0/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:43.970582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42206d15-a4c4-4697-9230-2ab77fa8d1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c72efd99534605b177aa144ebb5f38, user_id=9bf9c97c7917458297b90c7c13524cc7, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.973965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42206d15-a4c4-4697-9230-2ab77fa8d1b7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.978249 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c983d9c-e895-4199-bcfa-8dba8cd0ba0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=238b1450574c411682599a39ad0a8371, user_id=1aa023d6ce6b45efa086c40c292f4ad0, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:43.981187 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c983d9c-e895-4199-bcfa-8dba8cd0ba0b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:43.988205 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 833/1662] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:43 2022] PUT /identity/v3/projects/18c72efd99534605b177aa144ebb5f38/users/9bf9c97c7917458297b90c7c13524cc7/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:43.989155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42206d15-a4c4-4697-9230-2ab77fa8d1b7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:43.994097 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 830/1663] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:43 2022] PUT /identity/v3/projects/238b1450574c411682599a39ad0a8371/users/1aa023d6ce6b45efa086c40c292f4ad0/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:43.994914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c983d9c-e895-4199-bcfa-8dba8cd0ba0b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:44.055432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42206d15-a4c4-4697-9230-2ab77fa8d1b7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:44.060145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5beaa-987b-4dd2-a2ee-1bc7a9d5e4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:44.060300 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c983d9c-e895-4199-bcfa-8dba8cd0ba0b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:44.060880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5beaa-987b-4dd2-a2ee-1bc7a9d5e4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.061166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5beaa-987b-4dd2-a2ee-1bc7a9d5e4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.061438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5beaa-987b-4dd2-a2ee-1bc7a9d5e4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.062082 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5beaa-987b-4dd2-a2ee-1bc7a9d5e4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:44.063746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd5beaa-987b-4dd2-a2ee-1bc7a9d5e4f8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:44.066375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323af08-62db-47c7-b904-bba4fdfdbe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:44.067132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323af08-62db-47c7-b904-bba4fdfdbe47 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.068017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323af08-62db-47c7-b904-bba4fdfdbe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.068296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323af08-62db-47c7-b904-bba4fdfdbe47 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.068940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323af08-62db-47c7-b904-bba4fdfdbe47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:44.070512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323af08-62db-47c7-b904-bba4fdfdbe47 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:44.075140 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 834/1664] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:44.080127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58d305-7a27-49ab-b5d7-b2c3bbc84d1c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.080558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58d305-7a27-49ab-b5d7-b2c3bbc84d1c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.080850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58d305-7a27-49ab-b5d7-b2c3bbc84d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.085023 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 831/1665] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:44.086924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323af08-62db-47c7-b904-bba4fdfdbe47 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:44.100680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323af08-62db-47c7-b904-bba4fdfdbe47 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:44.102902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1f58d305-7a27-49ab-b5d7-b2c3bbc84d1c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:44.104342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1f58d305-7a27-49ab-b5d7-b2c3bbc84d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:44.105642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79146f-10d3-422f-8d1e-a4adba1afbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:44.106309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79146f-10d3-422f-8d1e-a4adba1afbee admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.106623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79146f-10d3-422f-8d1e-a4adba1afbee admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.106932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79146f-10d3-422f-8d1e-a4adba1afbee admin admin] PATH_INFO: `/v3/projects/bdc78df651ca400abd4b23f21cc2e45c/users/8f3767777868478ab2021a4510e80741/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.113457 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 835/1666] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:44.115964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58d305-7a27-49ab-b5d7-b2c3bbc84d1c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:44.122366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79146f-10d3-422f-8d1e-a4adba1afbee admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc78df651ca400abd4b23f21cc2e45c, user_id=8f3767777868478ab2021a4510e80741, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:44.125055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79146f-10d3-422f-8d1e-a4adba1afbee admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:44.132543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58d305-7a27-49ab-b5d7-b2c3bbc84d1c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:44.142794 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 832/1667] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:44 2022] PUT /identity/v3/projects/bdc78df651ca400abd4b23f21cc2e45c/users/8f3767777868478ab2021a4510e80741/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:44.143885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79146f-10d3-422f-8d1e-a4adba1afbee admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:44.145701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cd79146f-10d3-422f-8d1e-a4adba1afbee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:44.198195 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c7bf0-2e59-46f9-91fb-49e04a436b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:44.198967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c7bf0-2e59-46f9-91fb-49e04a436b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.199241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c7bf0-2e59-46f9-91fb-49e04a436b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.199509 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c7bf0-2e59-46f9-91fb-49e04a436b3b admin admin] PATH_INFO: `/v3/projects/18c72efd99534605b177aa144ebb5f38/users/9bf9c97c7917458297b90c7c13524cc7/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.201455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c7bf0-2e59-46f9-91fb-49e04a436b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c72efd99534605b177aa144ebb5f38, user_id=9bf9c97c7917458297b90c7c13524cc7, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:44.204185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c7bf0-2e59-46f9-91fb-49e04a436b3b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:44.216335 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 836/1668] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:44 2022] PUT /identity/v3/projects/18c72efd99534605b177aa144ebb5f38/users/9bf9c97c7917458297b90c7c13524cc7/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:44.217011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c7bf0-2e59-46f9-91fb-49e04a436b3b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:44.231098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79146f-10d3-422f-8d1e-a4adba1afbee admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:44.266276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:44.299830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9510407-1514-41a7-b4db-ae0a9f9526f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:44.300517 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9510407-1514-41a7-b4db-ae0a9f9526f6 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.300800 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9510407-1514-41a7-b4db-ae0a9f9526f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.301074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9510407-1514-41a7-b4db-ae0a9f9526f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.306503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:44.332207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c7bf0-2e59-46f9-91fb-49e04a436b3b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:44.337958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105fd12-d019-416a-9d71-51f76293a61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:44.338580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105fd12-d019-416a-9d71-51f76293a61c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.339067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105fd12-d019-416a-9d71-51f76293a61c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.339346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105fd12-d019-416a-9d71-51f76293a61c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.340469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105fd12-d019-416a-9d71-51f76293a61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:44.342601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105fd12-d019-416a-9d71-51f76293a61c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:44.354945 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 837/1669] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:44.356568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07209e0c-0ac8-446c-b495-2e5f7a1495a4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.356883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07209e0c-0ac8-446c-b495-2e5f7a1495a4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.357167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07209e0c-0ac8-446c-b495-2e5f7a1495a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.384828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-07209e0c-0ac8-446c-b495-2e5f7a1495a4 None None] MFA Rules not processed for user `9bf9c97c7917458297b90c7c13524cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:44.386049 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-07209e0c-0ac8-446c-b495-2e5f7a1495a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:44.391935 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9510407-1514-41a7-b4db-ae0a9f9526f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:44.393421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9510407-1514-41a7-b4db-ae0a9f9526f6 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:44.403081 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 833/1670] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:44.403913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9510407-1514-41a7-b4db-ae0a9f9526f6 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:44.413028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9510407-1514-41a7-b4db-ae0a9f9526f6 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:44.417953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e6ae2-58d7-4719-bfa9-95cd4f30f5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:44.418606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e6ae2-58d7-4719-bfa9-95cd4f30f5f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.418912 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e6ae2-58d7-4719-bfa9-95cd4f30f5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.419184 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e6ae2-58d7-4719-bfa9-95cd4f30f5f2 admin admin] PATH_INFO: `/v3/projects/bdc78df651ca400abd4b23f21cc2e45c/users/8f3767777868478ab2021a4510e80741/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.421078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e6ae2-58d7-4719-bfa9-95cd4f30f5f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc78df651ca400abd4b23f21cc2e45c, user_id=8f3767777868478ab2021a4510e80741, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:44.423721 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e6ae2-58d7-4719-bfa9-95cd4f30f5f2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:44.436398 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 834/1671] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:44 2022] PUT /identity/v3/projects/bdc78df651ca400abd4b23f21cc2e45c/users/8f3767777868478ab2021a4510e80741/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:44.457284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb97160-74e6-4f51-b925-a90ec9bc7519 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.458450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb97160-74e6-4f51-b925-a90ec9bc7519 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.459494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb97160-74e6-4f51-b925-a90ec9bc7519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.462363 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 838/1672] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:44.474519 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07209e0c-0ac8-446c-b495-2e5f7a1495a4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:44.475963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-07209e0c-0ac8-446c-b495-2e5f7a1495a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:44.505473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bfb97160-74e6-4f51-b925-a90ec9bc7519 None None] MFA Rules not processed for user `8f3767777868478ab2021a4510e80741`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:44.506981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bfb97160-74e6-4f51-b925-a90ec9bc7519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:44.555251 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07209e0c-0ac8-446c-b495-2e5f7a1495a4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:44.612322 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 835/1673] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:44.620430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cb59f-7c4b-4edf-95d3-553cbc2a7db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:44.621192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cb59f-7c4b-4edf-95d3-553cbc2a7db1 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.621473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cb59f-7c4b-4edf-95d3-553cbc2a7db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.621746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cb59f-7c4b-4edf-95d3-553cbc2a7db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.624087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ca3cb59f-7c4b-4edf-95d3-553cbc2a7db1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:44.625408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb97160-74e6-4f51-b925-a90ec9bc7519 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:44.627033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bfb97160-74e6-4f51-b925-a90ec9bc7519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:44.643772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb97160-74e6-4f51-b925-a90ec9bc7519 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:44.663566 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c66299-a5f8-48b6-87a3-e79fcd2409cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:44.665558 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c66299-a5f8-48b6-87a3-e79fcd2409cd service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:44.665846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c66299-a5f8-48b6-87a3-e79fcd2409cd service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:44.666522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c66299-a5f8-48b6-87a3-e79fcd2409cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:44.673434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:44.689584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c66299-a5f8-48b6-87a3-e79fcd2409cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:44.692580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c66299-a5f8-48b6-87a3-e79fcd2409cd service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:44.705973 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 836/1674] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:48:44.722166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cb59f-7c4b-4edf-95d3-553cbc2a7db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:44.723863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cb59f-7c4b-4edf-95d3-553cbc2a7db1 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:44.736143 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 839/1675] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:48:45.213317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c66299-a5f8-48b6-87a3-e79fcd2409cd service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:45.233380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c66299-a5f8-48b6-87a3-e79fcd2409cd service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:45.240873 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703142-1ce8-4aaa-a12f-9e1adbe61b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:45.243639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703142-1ce8-4aaa-a12f-9e1adbe61b85 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:45.244110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703142-1ce8-4aaa-a12f-9e1adbe61b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:45.246340 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703142-1ce8-4aaa-a12f-9e1adbe61b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:45.342234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:45.375253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf703142-1ce8-4aaa-a12f-9e1adbe61b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:45.381089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf703142-1ce8-4aaa-a12f-9e1adbe61b85 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:45.421476 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 837/1676] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:47.647703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cb59f-7c4b-4edf-95d3-553cbc2a7db1 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:47.656975 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cb59f-7c4b-4edf-95d3-553cbc2a7db1 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:47.661835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c070f9-54dd-4dec-92c0-e022f6b1fbdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:47.662526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c070f9-54dd-4dec-92c0-e022f6b1fbdd service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:47.662872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c070f9-54dd-4dec-92c0-e022f6b1fbdd service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:47.663598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c070f9-54dd-4dec-92c0-e022f6b1fbdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:47.770183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c070f9-54dd-4dec-92c0-e022f6b1fbdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:47.773268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c070f9-54dd-4dec-92c0-e022f6b1fbdd service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:47.784881 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 840/1677] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:48.343518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703142-1ce8-4aaa-a12f-9e1adbe61b85 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:48.456986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703142-1ce8-4aaa-a12f-9e1adbe61b85 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:48.469025 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:48.469693 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:48.469963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:48.470230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] PATH_INFO: `/v3/users/20284690f04a433188e0aefad57246e7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:48.472964 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] RBAC: Authorizing `identity:delete_user(user_id=20284690f04a433188e0aefad57246e7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:48.474995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:48.583473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c070f9-54dd-4dec-92c0-e022f6b1fbdd service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:48.597167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c070f9-54dd-4dec-92c0-e022f6b1fbdd service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:48.606600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7232f31-2608-4320-aee1-105e6fc81239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:48.607949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7232f31-2608-4320-aee1-105e6fc81239 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:48.608240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7232f31-2608-4320-aee1-105e6fc81239 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:48.608652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7232f31-2608-4320-aee1-105e6fc81239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:48.610366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20284690f04a433188e0aefad57246e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:48.611064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20284690f04a433188e0aefad57246e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:48.626247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20284690f04a433188e0aefad57246e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:48.642459 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20284690f04a433188e0aefad57246e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:48.663760 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-27e5e181-fd64-415f-8781-8cb14ba2227c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20284690f04a433188e0aefad57246e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:48.677023 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 838/1678] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:48 2022] DELETE /identity/v3/users/20284690f04a433188e0aefad57246e7 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:48.681968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24372e1-b1f5-4f04-95c8-dac798533aa0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:48.682395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24372e1-b1f5-4f04-95c8-dac798533aa0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:48.682814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24372e1-b1f5-4f04-95c8-dac798533aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:48.745731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7232f31-2608-4320-aee1-105e6fc81239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:48.759967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7232f31-2608-4320-aee1-105e6fc81239 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:48.797066 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 841/1679] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:48.798392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f24372e1-b1f5-4f04-95c8-dac798533aa0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:48.809171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f24372e1-b1f5-4f04-95c8-dac798533aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:48.939477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:48.962702 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 839/1680] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:48.976641 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7232f31-2608-4320-aee1-105e6fc81239 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:49.152623 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7232f31-2608-4320-aee1-105e6fc81239 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:49.184023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:49.242011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90423f-eaa3-4685-afff-16772b0a9945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:49.244120 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90423f-eaa3-4685-afff-16772b0a9945 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:49.244712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90423f-eaa3-4685-afff-16772b0a9945 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:49.245336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90423f-eaa3-4685-afff-16772b0a9945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:49.292355 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90423f-eaa3-4685-afff-16772b0a9945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:49.296575 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90423f-eaa3-4685-afff-16772b0a9945 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:49.325721 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 842/1681] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:49.628761 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24372e1-b1f5-4f04-95c8-dac798533aa0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:49.630195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f24372e1-b1f5-4f04-95c8-dac798533aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:49.666790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24372e1-b1f5-4f04-95c8-dac798533aa0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:49.673470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:49.673893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:49.674177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:49.674515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] PATH_INFO: `/v3/projects/8955229bb756458fb8202cae89a62243` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:49.676346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8955229bb756458fb8202cae89a62243)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:49.686063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:49.741928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8955229bb756458fb8202cae89a62243', 'request_id': 'req-8a79a3ba-0c0e-4e91-a055-1362f1028d65'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:49.755213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8955229bb756458fb8202cae89a62243', 'request_id': 'req-8a79a3ba-0c0e-4e91-a055-1362f1028d65'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:49.762449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8955229bb756458fb8202cae89a62243', 'request_id': 'req-8a79a3ba-0c0e-4e91-a055-1362f1028d65'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:49.778819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] The token cache is being invalidate because project 8955229bb756458fb8202cae89a62243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:49.779120 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:49.780934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:49.786030 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 840/1682] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:49 2022] DELETE /identity/v3/projects/8955229bb756458fb8202cae89a62243 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:49.786858 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90423f-eaa3-4685-afff-16772b0a9945 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:49.797936 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90423f-eaa3-4685-afff-16772b0a9945 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:49.875900 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:49.876806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:49.877223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:49.877769 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] PATH_INFO: `/v3/users/8f3767777868478ab2021a4510e80741` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:49.879109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3767777868478ab2021a4510e80741)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:49.881833 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:49.945399 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3767777868478ab2021a4510e80741'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:49.964395 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3767777868478ab2021a4510e80741'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:49.965541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3767777868478ab2021a4510e80741'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:49.975766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3767777868478ab2021a4510e80741'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:49.996239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b1282a4-a189-4e13-a15e-ede9f911c1d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3767777868478ab2021a4510e80741'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:50.003057 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 843/1683] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:49 2022] DELETE /identity/v3/users/8f3767777868478ab2021a4510e80741 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:51.182567 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:51.184068 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:51.202884 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79a3ba-0c0e-4e91-a055-1362f1028d65 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:51.343047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:51.343754 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:51.344549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:51.344997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] PATH_INFO: `/v3/projects/bdc78df651ca400abd4b23f21cc2e45c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:51.347066 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc78df651ca400abd4b23f21cc2e45c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:51.349401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:51.407285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc78df651ca400abd4b23f21cc2e45c', 'request_id': 'req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:51.439908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc78df651ca400abd4b23f21cc2e45c', 'request_id': 'req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:51.449749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc78df651ca400abd4b23f21cc2e45c', 'request_id': 'req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:51.469975 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] The token cache is being invalidate because project bdc78df651ca400abd4b23f21cc2e45c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:51.470560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:51.471462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c705aea6-7ad7-4304-bf5e-df84eff4bd6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:51.478556 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 841/1684] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:51 2022] DELETE /identity/v3/projects/bdc78df651ca400abd4b23f21cc2e45c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:51.479753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964b52-11be-4957-a369-6c6765c196c3 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:51.480281 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964b52-11be-4957-a369-6c6765c196c3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:51.480281 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964b52-11be-4957-a369-6c6765c196c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:51.504357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d7964b52-11be-4957-a369-6c6765c196c3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:51.505843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d7964b52-11be-4957-a369-6c6765c196c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:51.548670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:51.602320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0386b-5314-4b05-aab1-1b578f6fefff None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:51.602662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0386b-5314-4b05-aab1-1b578f6fefff None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:51.602904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0386b-5314-4b05-aab1-1b578f6fefff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:51.610704 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 844/1685] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:51.633266 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5ee0386b-5314-4b05-aab1-1b578f6fefff None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:51.634676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5ee0386b-5314-4b05-aab1-1b578f6fefff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:51.649445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7964b52-11be-4957-a369-6c6765c196c3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:51.660560 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 842/1686] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:51.664974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7964b52-11be-4957-a369-6c6765c196c3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:51.671551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba4f81-d7f4-4245-a07f-3adf0051cea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:51.672281 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba4f81-d7f4-4245-a07f-3adf0051cea5 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:51.672614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba4f81-d7f4-4245-a07f-3adf0051cea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:51.672886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba4f81-d7f4-4245-a07f-3adf0051cea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:51.673598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba4f81-d7f4-4245-a07f-3adf0051cea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:51.673884 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba4f81-d7f4-4245-a07f-3adf0051cea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:51.677638 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba4f81-d7f4-4245-a07f-3adf0051cea5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:51.699676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0386b-5314-4b05-aab1-1b578f6fefff None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:51.703543 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 845/1687] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:51.734802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0386b-5314-4b05-aab1-1b578f6fefff None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:51.752190 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84edc0-fd80-492f-99d7-b86d67057910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:51.752982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84edc0-fd80-492f-99d7-b86d67057910 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:51.753357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84edc0-fd80-492f-99d7-b86d67057910 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:51.754886 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84edc0-fd80-492f-99d7-b86d67057910 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:51.756676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84edc0-fd80-492f-99d7-b86d67057910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:51.764129 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84edc0-fd80-492f-99d7-b86d67057910 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:51.783008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:51.847803 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 843/1688] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:51 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:51.851777 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba4f81-d7f4-4245-a07f-3adf0051cea5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:51.884315 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba4f81-d7f4-4245-a07f-3adf0051cea5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:51.989424 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233a4af-a30d-476f-87ba-e4350d245426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:51.989737 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233a4af-a30d-476f-87ba-e4350d245426 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:51.990011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233a4af-a30d-476f-87ba-e4350d245426 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:51.990320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233a4af-a30d-476f-87ba-e4350d245426 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:51.990945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233a4af-a30d-476f-87ba-e4350d245426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:51.993565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233a4af-a30d-476f-87ba-e4350d245426 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:52.030195 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 846/1689] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:51 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:52.034229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84edc0-fd80-492f-99d7-b86d67057910 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:52.047621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84edc0-fd80-492f-99d7-b86d67057910 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:52.052522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f015e9c-11ab-4a49-b1cb-8ce59ba056b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:52.053128 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f015e9c-11ab-4a49-b1cb-8ce59ba056b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:52.053400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f015e9c-11ab-4a49-b1cb-8ce59ba056b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:52.053663 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f015e9c-11ab-4a49-b1cb-8ce59ba056b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:52.054300 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f015e9c-11ab-4a49-b1cb-8ce59ba056b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:52.055897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f015e9c-11ab-4a49-b1cb-8ce59ba056b8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:52.067618 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 844/1690] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:52.071622 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233a4af-a30d-476f-87ba-e4350d245426 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:52.085226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233a4af-a30d-476f-87ba-e4350d245426 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:52.093314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf75db4-5a16-4156-99f4-1791edacd18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:52.093947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf75db4-5a16-4156-99f4-1791edacd18f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:52.094246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf75db4-5a16-4156-99f4-1791edacd18f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:52.094547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf75db4-5a16-4156-99f4-1791edacd18f admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:52.095331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf75db4-5a16-4156-99f4-1791edacd18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:52.097332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf75db4-5a16-4156-99f4-1791edacd18f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:52.114705 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 847/1691] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:52.115125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f015e9c-11ab-4a49-b1cb-8ce59ba056b8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:52.130843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f015e9c-11ab-4a49-b1cb-8ce59ba056b8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:52.136110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9d26a-caac-4c01-8a0e-2a46123ee63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:52.136824 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9d26a-caac-4c01-8a0e-2a46123ee63c admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:52.137101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9d26a-caac-4c01-8a0e-2a46123ee63c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:52.137372 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9d26a-caac-4c01-8a0e-2a46123ee63c admin admin] PATH_INFO: `/v3/projects/c5f2240b2c6a43b8a810d84065ec9d44/users/bee1c05fbfb94433a97928fa2eb5cd15/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:52.155340 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9d26a-caac-4c01-8a0e-2a46123ee63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f2240b2c6a43b8a810d84065ec9d44, user_id=bee1c05fbfb94433a97928fa2eb5cd15, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:52.158113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9d26a-caac-4c01-8a0e-2a46123ee63c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:52.184085 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 845/1692] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:52 2022] PUT /identity/v3/projects/c5f2240b2c6a43b8a810d84065ec9d44/users/bee1c05fbfb94433a97928fa2eb5cd15/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:52.189333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16bd8f-a004-4a0d-ac28-e84dfb61b94e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:52.189777 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16bd8f-a004-4a0d-ac28-e84dfb61b94e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:52.190472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16bd8f-a004-4a0d-ac28-e84dfb61b94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:52.228749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2a16bd8f-a004-4a0d-ac28-e84dfb61b94e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:52.230191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2a16bd8f-a004-4a0d-ac28-e84dfb61b94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:52.329573 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 848/1693] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:52.339008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9d26a-caac-4c01-8a0e-2a46123ee63c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:52.343631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d2c9d26a-caac-4c01-8a0e-2a46123ee63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:52.437910 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9d26a-caac-4c01-8a0e-2a46123ee63c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:52.442310 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357e6dc-fea7-4bc6-b7a8-712af021a128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:52.443369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357e6dc-fea7-4bc6-b7a8-712af021a128 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:52.443721 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357e6dc-fea7-4bc6-b7a8-712af021a128 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:52.444061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357e6dc-fea7-4bc6-b7a8-712af021a128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:52.462981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357e6dc-fea7-4bc6-b7a8-712af021a128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:52.464457 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357e6dc-fea7-4bc6-b7a8-712af021a128 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:52.475908 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 846/1694] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:52.687430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd368b-1482-4636-9420-e8471ec54740 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:52.687970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd368b-1482-4636-9420-e8471ec54740 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:52.687970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd368b-1482-4636-9420-e8471ec54740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:52.763338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-20cd368b-1482-4636-9420-e8471ec54740 None None] MFA Rules not processed for user `1aa023d6ce6b45efa086c40c292f4ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:52.765510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-20cd368b-1482-4636-9420-e8471ec54740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:52.849761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:52.904364 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 849/1695] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:52.905305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357e6dc-fea7-4bc6-b7a8-712af021a128 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:52.906677 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1357e6dc-fea7-4bc6-b7a8-712af021a128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:52.932084 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357e6dc-fea7-4bc6-b7a8-712af021a128 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:52.946000 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e3284-feb7-44d9-b447-126a00c1dbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:52.946740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e3284-feb7-44d9-b447-126a00c1dbe1 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:52.947044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e3284-feb7-44d9-b447-126a00c1dbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:52.947361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e3284-feb7-44d9-b447-126a00c1dbe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:52.948840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e3284-feb7-44d9-b447-126a00c1dbe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:52.950817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e3284-feb7-44d9-b447-126a00c1dbe1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:53.019655 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 847/1696] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:52 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:48:53.023816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd368b-1482-4636-9420-e8471ec54740 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:53.036937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd368b-1482-4636-9420-e8471ec54740 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:53.205301 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc2ffa-2ccc-429e-bcee-8c54b77f99ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:53.205961 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc2ffa-2ccc-429e-bcee-8c54b77f99ab admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:53.206238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc2ffa-2ccc-429e-bcee-8c54b77f99ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:53.206746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc2ffa-2ccc-429e-bcee-8c54b77f99ab admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:53.207456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc2ffa-2ccc-429e-bcee-8c54b77f99ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:53.209413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc2ffa-2ccc-429e-bcee-8c54b77f99ab admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:53.272667 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 850/1697] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:53 2022] POST /identity/v3/users => generated 298 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:53.276568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e3284-feb7-44d9-b447-126a00c1dbe1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:53.297196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e3284-feb7-44d9-b447-126a00c1dbe1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:53.302782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f5a35-fb49-4887-bf37-9edad2c4cd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:53.303608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f5a35-fb49-4887-bf37-9edad2c4cd0b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:53.304797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f5a35-fb49-4887-bf37-9edad2c4cd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:53.305380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f5a35-fb49-4887-bf37-9edad2c4cd0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:53.306656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f5a35-fb49-4887-bf37-9edad2c4cd0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:53.308460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f5a35-fb49-4887-bf37-9edad2c4cd0b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:53.328090 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 848/1698] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:53.330001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc2ffa-2ccc-429e-bcee-8c54b77f99ab admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:53.346036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc2ffa-2ccc-429e-bcee-8c54b77f99ab admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:53.354271 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb471612-318e-405a-b3c5-4e53eca91ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:53.355049 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb471612-318e-405a-b3c5-4e53eca91ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:53.355547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb471612-318e-405a-b3c5-4e53eca91ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:53.355657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb471612-318e-405a-b3c5-4e53eca91ee6 admin admin] PATH_INFO: `/v3/projects/0043aeec1ea646f3be919354aaa96452/users/e89c572bdd364c0b9c9dc1b843daffdf/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:53.398648 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb471612-318e-405a-b3c5-4e53eca91ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0043aeec1ea646f3be919354aaa96452, user_id=e89c572bdd364c0b9c9dc1b843daffdf, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:53.401302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb471612-318e-405a-b3c5-4e53eca91ee6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:53.426765 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 851/1699] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:53 2022] PUT /identity/v3/projects/0043aeec1ea646f3be919354aaa96452/users/e89c572bdd364c0b9c9dc1b843daffdf/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:53.431437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f5a35-fb49-4887-bf37-9edad2c4cd0b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:53.454077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f5a35-fb49-4887-bf37-9edad2c4cd0b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:53.544690 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb471612-318e-405a-b3c5-4e53eca91ee6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:53.546787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-eb471612-318e-405a-b3c5-4e53eca91ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:53.592217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302359a-8b16-46bb-8600-6a6a6e69e584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:53.592849 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302359a-8b16-46bb-8600-6a6a6e69e584 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:53.593426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302359a-8b16-46bb-8600-6a6a6e69e584 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:53.593704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302359a-8b16-46bb-8600-6a6a6e69e584 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:53.594361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302359a-8b16-46bb-8600-6a6a6e69e584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:53.596270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302359a-8b16-46bb-8600-6a6a6e69e584 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:53.619769 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 849/1700] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:53.620995 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302359a-8b16-46bb-8600-6a6a6e69e584 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:53.634895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302359a-8b16-46bb-8600-6a6a6e69e584 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:53.642097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f58ae3-62c2-4731-86ce-7b3e87aaf497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:53.642901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58ae3-62c2-4731-86ce-7b3e87aaf497 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:53.643263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58ae3-62c2-4731-86ce-7b3e87aaf497 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:53.643586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f58ae3-62c2-4731-86ce-7b3e87aaf497 admin admin] PATH_INFO: `/v3/projects/0043aeec1ea646f3be919354aaa96452/users/e89c572bdd364c0b9c9dc1b843daffdf/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:53.645791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f58ae3-62c2-4731-86ce-7b3e87aaf497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0043aeec1ea646f3be919354aaa96452, user_id=e89c572bdd364c0b9c9dc1b843daffdf, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:53.649103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f58ae3-62c2-4731-86ce-7b3e87aaf497 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:53.686924 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 850/1701] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:53 2022] PUT /identity/v3/projects/0043aeec1ea646f3be919354aaa96452/users/e89c572bdd364c0b9c9dc1b843daffdf/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:53.697901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb471612-318e-405a-b3c5-4e53eca91ee6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:53.717693 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd057e-c942-4cc4-9349-43ac50cf4e85 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:53.718213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd057e-c942-4cc4-9349-43ac50cf4e85 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:53.718776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd057e-c942-4cc4-9349-43ac50cf4e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:53.730736 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:53.782026 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-64fd057e-c942-4cc4-9349-43ac50cf4e85 None None] MFA Rules not processed for user `bee1c05fbfb94433a97928fa2eb5cd15`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:53.783636 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-64fd057e-c942-4cc4-9349-43ac50cf4e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:53.808363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:53.842478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:53.843900 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:53.843900 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:53.844124 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] PATH_INFO: `/v3/users/6251b7909a7b43a0bec0de0392aee1e0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:53.845159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6251b7909a7b43a0bec0de0392aee1e0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:53.847240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:53.906897 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 851/1702] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:53.917154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd057e-c942-4cc4-9349-43ac50cf4e85 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:53.918708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-64fd057e-c942-4cc4-9349-43ac50cf4e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:53.920901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6251b7909a7b43a0bec0de0392aee1e0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:53.944683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6251b7909a7b43a0bec0de0392aee1e0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:53.946780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6251b7909a7b43a0bec0de0392aee1e0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:53.960002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6251b7909a7b43a0bec0de0392aee1e0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:53.995916 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98557e71-12df-4d3e-bd14-cda27a9c8124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6251b7909a7b43a0bec0de0392aee1e0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:54.004621 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 852/1703] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:53 2022] DELETE /identity/v3/users/6251b7909a7b43a0bec0de0392aee1e0 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:48:54.008640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eda23-364b-4038-a858-bc281c661c15 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:54.009081 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eda23-364b-4038-a858-bc281c661c15 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:54.009514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632eda23-364b-4038-a858-bc281c661c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:54.015816 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd057e-c942-4cc4-9349-43ac50cf4e85 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:54.032012 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-632eda23-364b-4038-a858-bc281c661c15 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:54.033477 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-632eda23-364b-4038-a858-bc281c661c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:54.116889 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 853/1704] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:54.130411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632eda23-364b-4038-a858-bc281c661c15 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:54.135291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:54.135942 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:54.136253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:54.136535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:54.140176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-632eda23-364b-4038-a858-bc281c661c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:54.147327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:54.243275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:54.245153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:54.294015 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632eda23-364b-4038-a858-bc281c661c15 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:54.295735 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 852/1705] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:54.306326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec745829-10e0-4b00-994a-af863c76006c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:54.307056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec745829-10e0-4b00-994a-af863c76006c service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:54.307340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec745829-10e0-4b00-994a-af863c76006c service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:54.307611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec745829-10e0-4b00-994a-af863c76006c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:54.325951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec745829-10e0-4b00-994a-af863c76006c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:54.328417 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec745829-10e0-4b00-994a-af863c76006c service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:54.355959 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 854/1706] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:48:54.786889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:54.795847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:54.802761 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:54.812769 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b33cc-50a5-4739-964c-b2c0a10004a4 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:54.829492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:54.831281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:54.831900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:54.832059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] PATH_INFO: `/v3/projects/d8cb97e6993049e0840e291e83f3a55f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:54.834018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8cb97e6993049e0840e291e83f3a55f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:54.836160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:54.918619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8cb97e6993049e0840e291e83f3a55f', 'request_id': 'req-1c046cc4-606b-42f1-9864-08434a611d34'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:54.924937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8cb97e6993049e0840e291e83f3a55f', 'request_id': 'req-1c046cc4-606b-42f1-9864-08434a611d34'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:54.932328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8cb97e6993049e0840e291e83f3a55f', 'request_id': 'req-1c046cc4-606b-42f1-9864-08434a611d34'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:54.952732 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] The token cache is being invalidate because project d8cb97e6993049e0840e291e83f3a55f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:54.953024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:54.953768 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1c046cc4-606b-42f1-9864-08434a611d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:54.955638 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 853/1707] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:54 2022] DELETE /identity/v3/projects/d8cb97e6993049e0840e291e83f3a55f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:54.959597 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec745829-10e0-4b00-994a-af863c76006c service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:54.971700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec745829-10e0-4b00-994a-af863c76006c service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:54.980333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:55.065090 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:55.066071 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:55.066523 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:55.067455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] PATH_INFO: `/v3/users/43e7770fc3d64657b7a716fe5ac02f6c` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:55.068806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e7770fc3d64657b7a716fe5ac02f6c)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:55.070939 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:55.133453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e7770fc3d64657b7a716fe5ac02f6c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:55.145176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e7770fc3d64657b7a716fe5ac02f6c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:55.146615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e7770fc3d64657b7a716fe5ac02f6c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:55.166333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e7770fc3d64657b7a716fe5ac02f6c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:55.184378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e7770fc3d64657b7a716fe5ac02f6c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:55.191776 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 855/1708] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:54 2022] DELETE /identity/v3/users/43e7770fc3d64657b7a716fe5ac02f6c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:55.974303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a02f3-edd0-4be6-88d5-134ee113a485 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:55.974769 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a02f3-edd0-4be6-88d5-134ee113a485 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:55.974769 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a02f3-edd0-4be6-88d5-134ee113a485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:55.995584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-765a02f3-edd0-4be6-88d5-134ee113a485 None None] MFA Rules not processed for user `bee1c05fbfb94433a97928fa2eb5cd15`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:55.996833 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-765a02f3-edd0-4be6-88d5-134ee113a485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:56.085951 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 854/1709] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:56.526579 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:56.527940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:56.768835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231da7f1-39ce-4a45-a6b4-371109abd88d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:56.774217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:56.775135 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:56.775410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:56.775685 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] PATH_INFO: `/v3/projects/659fad43bc3347fc962a899a4641b7da` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:56.777759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=659fad43bc3347fc962a899a4641b7da)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:56.791072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:56.898879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:56.928259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659fad43bc3347fc962a899a4641b7da', 'request_id': 'req-215faddb-6843-4bb6-bb40-2303a79720d3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:56.951890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659fad43bc3347fc962a899a4641b7da', 'request_id': 'req-215faddb-6843-4bb6-bb40-2303a79720d3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:56.964025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659fad43bc3347fc962a899a4641b7da', 'request_id': 'req-215faddb-6843-4bb6-bb40-2303a79720d3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:56.993398 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] The token cache is being invalidate because project 659fad43bc3347fc962a899a4641b7da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:56.994067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:56.995221 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:56.997032 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 856/1710] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:56 2022] DELETE /identity/v3/projects/659fad43bc3347fc962a899a4641b7da => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:57.001347 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a02f3-edd0-4be6-88d5-134ee113a485 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:57.015857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a02f3-edd0-4be6-88d5-134ee113a485 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:57.107625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:57.108670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:57.109131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:57.109629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] PATH_INFO: `/v3/users/9bf9c97c7917458297b90c7c13524cc7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:57.111088 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf9c97c7917458297b90c7c13524cc7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:57.113310 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:57.164997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:57.191306 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf9c97c7917458297b90c7c13524cc7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:57.192108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf9c97c7917458297b90c7c13524cc7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:57.201978 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf9c97c7917458297b90c7c13524cc7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:57.212918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf9c97c7917458297b90c7c13524cc7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:57.235670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a92ffa-9cc9-4c67-b502-e2368e9f5bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf9c97c7917458297b90c7c13524cc7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:57.246299 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 855/1711] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:48:57 2022] DELETE /identity/v3/users/9bf9c97c7917458297b90c7c13524cc7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:58.655318 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:58.657464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:58.688918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215faddb-6843-4bb6-bb40-2303a79720d3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:58.819546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:58.820407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:58.820691 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:58.821159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] PATH_INFO: `/v3/projects/18c72efd99534605b177aa144ebb5f38` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:58.823526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18c72efd99534605b177aa144ebb5f38)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:58.830145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:58.982759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18c72efd99534605b177aa144ebb5f38', 'request_id': 'req-c50306b1-b4be-4d54-b567-b4d26afa6c30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:58.996050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18c72efd99534605b177aa144ebb5f38', 'request_id': 'req-c50306b1-b4be-4d54-b567-b4d26afa6c30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:59.006302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18c72efd99534605b177aa144ebb5f38', 'request_id': 'req-c50306b1-b4be-4d54-b567-b4d26afa6c30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:59.057908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] The token cache is being invalidate because project 18c72efd99534605b177aa144ebb5f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:48:59.058248 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:59.059109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c50306b1-b4be-4d54-b567-b4d26afa6c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:48:59.060747 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 857/1712] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:48:58 2022] DELETE /identity/v3/projects/18c72efd99534605b177aa144ebb5f38 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:59.067796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0a2a-2ec9-4a32-84f3-775d35e3649d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:59.068325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0a2a-2ec9-4a32-84f3-775d35e3649d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:59.068325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e0a2a-2ec9-4a32-84f3-775d35e3649d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:59.112630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e92e0a2a-2ec9-4a32-84f3-775d35e3649d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:59.116030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e92e0a2a-2ec9-4a32-84f3-775d35e3649d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:59.286625 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 856/1713] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:59.295428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6f401-12bf-413c-a91b-ff4867d67b3f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:59.295715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6f401-12bf-413c-a91b-ff4867d67b3f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:59.295995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6f401-12bf-413c-a91b-ff4867d67b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:59.338639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-90d6f401-12bf-413c-a91b-ff4867d67b3f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:59.339921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-90d6f401-12bf-413c-a91b-ff4867d67b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:59.372707 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 858/1714] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:48:59.376676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e0a2a-2ec9-4a32-84f3-775d35e3649d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:59.395296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e0a2a-2ec9-4a32-84f3-775d35e3649d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:59.403685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d27c2-acfe-497e-a232-372bc1ccb626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:59.404478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d27c2-acfe-497e-a232-372bc1ccb626 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:59.404914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d27c2-acfe-497e-a232-372bc1ccb626 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:59.405338 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d27c2-acfe-497e-a232-372bc1ccb626 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:59.406150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d27c2-acfe-497e-a232-372bc1ccb626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:48:59.406601 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d27c2-acfe-497e-a232-372bc1ccb626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:59.408394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d27c2-acfe-497e-a232-372bc1ccb626 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:59.440396 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 857/1715] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:48:59.445232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6f401-12bf-413c-a91b-ff4867d67b3f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:59.462355 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6f401-12bf-413c-a91b-ff4867d67b3f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:59.469659 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b46e39-e43f-47ec-8c72-b334b79797ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:59.479104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b46e39-e43f-47ec-8c72-b334b79797ef admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:59.479809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b46e39-e43f-47ec-8c72-b334b79797ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:59.480231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b46e39-e43f-47ec-8c72-b334b79797ef admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:59.481264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b46e39-e43f-47ec-8c72-b334b79797ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:59.484031 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b46e39-e43f-47ec-8c72-b334b79797ef admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:59.540159 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 859/1716] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:48:59 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:48:59.544330 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d27c2-acfe-497e-a232-372bc1ccb626 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:59.557066 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d27c2-acfe-497e-a232-372bc1ccb626 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:59.692091 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a235e8-2788-41c2-b3f3-cc3f5919a840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:59.692934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a235e8-2788-41c2-b3f3-cc3f5919a840 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:59.693285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a235e8-2788-41c2-b3f3-cc3f5919a840 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:59.693642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a235e8-2788-41c2-b3f3-cc3f5919a840 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:59.694345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a235e8-2788-41c2-b3f3-cc3f5919a840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:59.696498 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a235e8-2788-41c2-b3f3-cc3f5919a840 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:59.732166 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 858/1717] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:48:59 2022] POST /identity/v3/users => generated 283 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:48:59.735923 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b46e39-e43f-47ec-8c72-b334b79797ef admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:59.751231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b46e39-e43f-47ec-8c72-b334b79797ef admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:59.762123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc9028-a652-48c0-8893-e5c05de28d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:59.762792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc9028-a652-48c0-8893-e5c05de28d71 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:59.763241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc9028-a652-48c0-8893-e5c05de28d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:59.763519 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc9028-a652-48c0-8893-e5c05de28d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:59.764183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc9028-a652-48c0-8893-e5c05de28d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:59.766064 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc9028-a652-48c0-8893-e5c05de28d71 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:59.788773 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 860/1718] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:59.792954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a235e8-2788-41c2-b3f3-cc3f5919a840 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:59.806035 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a235e8-2788-41c2-b3f3-cc3f5919a840 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:59.813043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfce2e9-a884-4a3d-b284-8e6cea318783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:59.813879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfce2e9-a884-4a3d-b284-8e6cea318783 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:59.814264 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfce2e9-a884-4a3d-b284-8e6cea318783 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:59.814640 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfce2e9-a884-4a3d-b284-8e6cea318783 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:59.815450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfce2e9-a884-4a3d-b284-8e6cea318783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:59.817768 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfce2e9-a884-4a3d-b284-8e6cea318783 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:59.830881 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 859/1719] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:48:59.837842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc9028-a652-48c0-8893-e5c05de28d71 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:48:59.849449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc9028-a652-48c0-8893-e5c05de28d71 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:48:59.858428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4246f4-6afe-4da1-8010-185c3edc0d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:48:59.860087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4246f4-6afe-4da1-8010-185c3edc0d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:59.862136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4246f4-6afe-4da1-8010-185c3edc0d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:59.862667 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4246f4-6afe-4da1-8010-185c3edc0d4f admin admin] PATH_INFO: `/v3/projects/3d640d2de11941d8b5170d141fde1209/users/62d26bfde62b4fe68851ddf959e4b4d5/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:59.874179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:48:59.882207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4246f4-6afe-4da1-8010-185c3edc0d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d640d2de11941d8b5170d141fde1209, user_id=62d26bfde62b4fe68851ddf959e4b4d5, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:48:59.885788 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4246f4-6afe-4da1-8010-185c3edc0d4f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:48:59.910760 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 861/1720] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:48:59 2022] PUT /identity/v3/projects/3d640d2de11941d8b5170d141fde1209/users/62d26bfde62b4fe68851ddf959e4b4d5/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:48:59.918061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b992458-b755-4ee1-9917-7cc613713b41 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:48:59.918521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b992458-b755-4ee1-9917-7cc613713b41 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:48:59.919098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b992458-b755-4ee1-9917-7cc613713b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:48:59.953063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4b992458-b755-4ee1-9917-7cc613713b41 None None] MFA Rules not processed for user `62d26bfde62b4fe68851ddf959e4b4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:48:59.954430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4b992458-b755-4ee1-9917-7cc613713b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:48:59.959341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:00.043323 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 860/1721] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:48:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:00.056299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4246f4-6afe-4da1-8010-185c3edc0d4f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:00.058203 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cc4246f4-6afe-4da1-8010-185c3edc0d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:00.142260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4246f4-6afe-4da1-8010-185c3edc0d4f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:00.150784 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83f443-a2ce-4330-9dc6-217e0229f3c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:00.151681 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83f443-a2ce-4330-9dc6-217e0229f3c3 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:00.152247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83f443-a2ce-4330-9dc6-217e0229f3c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:00.152960 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83f443-a2ce-4330-9dc6-217e0229f3c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:00.171115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83f443-a2ce-4330-9dc6-217e0229f3c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:00.172831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83f443-a2ce-4330-9dc6-217e0229f3c3 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:00.199592 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 862/1722] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:02.237215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ae958-c804-4fd7-9132-05d195b258da None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:02.237707 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ae958-c804-4fd7-9132-05d195b258da None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:02.237906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ae958-c804-4fd7-9132-05d195b258da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:02.302653 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-255ae958-c804-4fd7-9132-05d195b258da None None] MFA Rules not processed for user `e89c572bdd364c0b9c9dc1b843daffdf`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:02.304537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-255ae958-c804-4fd7-9132-05d195b258da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:02.408429 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 861/1723] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:02.425074 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83f443-a2ce-4330-9dc6-217e0229f3c3 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:02.426781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8a83f443-a2ce-4330-9dc6-217e0229f3c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:02.508708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83f443-a2ce-4330-9dc6-217e0229f3c3 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:02.516564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b1c2ea68e4741728469ab7cc542b6d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:02.517225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:02.517504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:02.517775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:02.520163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:02.638626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:02.640318 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:02.657612 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 863/1724] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:03.047419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96f98a-36ec-4400-82d8-6bda61ae2c39 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:03.047960 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96f98a-36ec-4400-82d8-6bda61ae2c39 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:03.048440 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96f98a-36ec-4400-82d8-6bda61ae2c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:03.076406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3a96f98a-36ec-4400-82d8-6bda61ae2c39 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:03.078121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3a96f98a-36ec-4400-82d8-6bda61ae2c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:03.159702 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 862/1725] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:49:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:03.252798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:03.254842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:03.348876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:03.385771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4aec-8d5f-4db6-b2d5-4667f4c88022 service glance] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:03.393090 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706ddf9-35b3-4b95-a46f-a055b4f7df2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:03.395525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706ddf9-35b3-4b95-a46f-a055b4f7df2b service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:03.396038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706ddf9-35b3-4b95-a46f-a055b4f7df2b service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:03.396479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706ddf9-35b3-4b95-a46f-a055b4f7df2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:03.417609 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706ddf9-35b3-4b95-a46f-a055b4f7df2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:03.419898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706ddf9-35b3-4b95-a46f-a055b4f7df2b service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:03.431496 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 864/1726] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:49:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:49:05.538129 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96f98a-36ec-4400-82d8-6bda61ae2c39 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:05.540398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3a96f98a-36ec-4400-82d8-6bda61ae2c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:05.685365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:05.748831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96f98a-36ec-4400-82d8-6bda61ae2c39 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:05.757780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:05.758904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:05.759197 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:05.759478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] PATH_INFO: `/v3/users/b002748b80db40358571017d6b42ed09` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:05.761308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b002748b80db40358571017d6b42ed09)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:05.764314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:05.837561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b002748b80db40358571017d6b42ed09'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:05.846831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b002748b80db40358571017d6b42ed09'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:05.861900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b002748b80db40358571017d6b42ed09'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:05.874309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b002748b80db40358571017d6b42ed09'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:05.891241 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-01f4f916-c69c-4fb6-934a-42181a2fac74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b002748b80db40358571017d6b42ed09'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:05.904703 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 863/1727] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:49:05 2022] DELETE /identity/v3/users/b002748b80db40358571017d6b42ed09 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:06.139675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706ddf9-35b3-4b95-a46f-a055b4f7df2b service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:06.159883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706ddf9-35b3-4b95-a46f-a055b4f7df2b service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:06.274431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:06.275130 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:06.275411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:06.275688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] PATH_INFO: `/v3/projects/7fa4a0e9972c4eb1963334ba4ae99a4f` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:06.277146 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fa4a0e9972c4eb1963334ba4ae99a4f)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:06.279156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:06.360967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fa4a0e9972c4eb1963334ba4ae99a4f', 'request_id': 'req-98c08028-4cea-4763-b5eb-b87a536792bf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:06.368976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fa4a0e9972c4eb1963334ba4ae99a4f', 'request_id': 'req-98c08028-4cea-4763-b5eb-b87a536792bf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:06.376978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fa4a0e9972c4eb1963334ba4ae99a4f', 'request_id': 'req-98c08028-4cea-4763-b5eb-b87a536792bf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:06.401665 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] The token cache is being invalidate because project 7fa4a0e9972c4eb1963334ba4ae99a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:49:06.401971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:06.403297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-98c08028-4cea-4763-b5eb-b87a536792bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:06.405529 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 865/1728] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:49:06 2022] DELETE /identity/v3/projects/7fa4a0e9972c4eb1963334ba4ae99a4f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:06.410493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5a48f-00b7-4cf9-9c1e-c2003208233d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:06.410947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5a48f-00b7-4cf9-9c1e-c2003208233d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:06.411324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5a48f-00b7-4cf9-9c1e-c2003208233d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:06.436389 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-94c5a48f-00b7-4cf9-9c1e-c2003208233d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:06.437796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-94c5a48f-00b7-4cf9-9c1e-c2003208233d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:06.533082 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 864/1729] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:06.534806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e3512-4bff-49ff-b265-ccf6a674b60a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:06.535101 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e3512-4bff-49ff-b265-ccf6a674b60a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:06.535482 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e3512-4bff-49ff-b265-ccf6a674b60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:06.569061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c20e3512-4bff-49ff-b265-ccf6a674b60a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:06.570503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c20e3512-4bff-49ff-b265-ccf6a674b60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:06.597817 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 866/1730] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:06.601765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5a48f-00b7-4cf9-9c1e-c2003208233d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:06.619614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5a48f-00b7-4cf9-9c1e-c2003208233d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:06.644453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea7f6b-8bee-4ade-b305-c546f8c2b973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:06.645141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7f6b-8bee-4ade-b305-c546f8c2b973 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:06.645432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7f6b-8bee-4ade-b305-c546f8c2b973 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:06.645729 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7f6b-8bee-4ade-b305-c546f8c2b973 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:06.646395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea7f6b-8bee-4ade-b305-c546f8c2b973 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:49:06.646710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea7f6b-8bee-4ade-b305-c546f8c2b973 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:06.648391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea7f6b-8bee-4ade-b305-c546f8c2b973 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:06.682680 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 865/1731] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:49:06.683923 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e3512-4bff-49ff-b265-ccf6a674b60a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:06.695597 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e3512-4bff-49ff-b265-ccf6a674b60a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:06.710143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96b41c-ef7f-47f3-81ba-f163a543d40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:06.710827 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b41c-ef7f-47f3-81ba-f163a543d40b admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:06.711114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b41c-ef7f-47f3-81ba-f163a543d40b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:06.711385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96b41c-ef7f-47f3-81ba-f163a543d40b admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:06.712036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96b41c-ef7f-47f3-81ba-f163a543d40b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:06.714094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96b41c-ef7f-47f3-81ba-f163a543d40b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:06.773423 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 867/1732] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:49:06 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:06.774448 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea7f6b-8bee-4ade-b305-c546f8c2b973 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:06.785887 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea7f6b-8bee-4ade-b305-c546f8c2b973 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:06.895911 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:06.926957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269abc1-18a4-4886-9cf3-3479d20a0c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:06.928324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269abc1-18a4-4886-9cf3-3479d20a0c17 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:06.928625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269abc1-18a4-4886-9cf3-3479d20a0c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:06.928901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269abc1-18a4-4886-9cf3-3479d20a0c17 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:06.929575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269abc1-18a4-4886-9cf3-3479d20a0c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:06.932600 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269abc1-18a4-4886-9cf3-3479d20a0c17 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:06.966785 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 866/1733] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:49:06 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:06.968771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96b41c-ef7f-47f3-81ba-f163a543d40b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:06.982766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96b41c-ef7f-47f3-81ba-f163a543d40b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:06.988446 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c56fed-7634-42fa-a530-91b3abe11a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:06.989963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c56fed-7634-42fa-a530-91b3abe11a15 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:06.990466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c56fed-7634-42fa-a530-91b3abe11a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:06.990973 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c56fed-7634-42fa-a530-91b3abe11a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:06.992018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c56fed-7634-42fa-a530-91b3abe11a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:07.000538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c56fed-7634-42fa-a530-91b3abe11a15 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:07.024195 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 868/1734] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:07.024842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269abc1-18a4-4886-9cf3-3479d20a0c17 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:07.038089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269abc1-18a4-4886-9cf3-3479d20a0c17 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:07.044391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4b726a-0102-4286-b3bf-fa5c0119cd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:07.049286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b726a-0102-4286-b3bf-fa5c0119cd9c admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:07.050188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b726a-0102-4286-b3bf-fa5c0119cd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:07.051431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b726a-0102-4286-b3bf-fa5c0119cd9c admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:07.054557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b726a-0102-4286-b3bf-fa5c0119cd9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:07.056598 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4b726a-0102-4286-b3bf-fa5c0119cd9c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:07.084125 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 867/1735] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:49:07.086156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c56fed-7634-42fa-a530-91b3abe11a15 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:07.114260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c56fed-7634-42fa-a530-91b3abe11a15 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:07.121900 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fd10e-afaf-44d5-a71b-0c32b837310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:07.122627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fd10e-afaf-44d5-a71b-0c32b837310c admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:07.122930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fd10e-afaf-44d5-a71b-0c32b837310c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:07.123207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fd10e-afaf-44d5-a71b-0c32b837310c admin admin] PATH_INFO: `/v3/projects/0001092a700c4a76bbbfd678b6df7e51/users/365f391239574744be58dc5cd4ef986c/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:07.146107 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fd10e-afaf-44d5-a71b-0c32b837310c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0001092a700c4a76bbbfd678b6df7e51, user_id=365f391239574744be58dc5cd4ef986c, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:07.150008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fd10e-afaf-44d5-a71b-0c32b837310c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:07.179314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:07.203864 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 869/1736] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:07 2022] PUT /identity/v3/projects/0001092a700c4a76bbbfd678b6df7e51/users/365f391239574744be58dc5cd4ef986c/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:07.214111 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0c662-0742-42cc-ae57-fb0d1a0ac81c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:07.214396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0c662-0742-42cc-ae57-fb0d1a0ac81c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:07.215177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0c662-0742-42cc-ae57-fb0d1a0ac81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:07.255135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-63a0c662-0742-42cc-ae57-fb0d1a0ac81c None None] MFA Rules not processed for user `365f391239574744be58dc5cd4ef986c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:07.256376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-63a0c662-0742-42cc-ae57-fb0d1a0ac81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:07.493516 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 868/1737] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:49:07.513406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fd10e-afaf-44d5-a71b-0c32b837310c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:07.515734 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8a8fd10e-afaf-44d5-a71b-0c32b837310c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:07.625683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fd10e-afaf-44d5-a71b-0c32b837310c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:07.633077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262695f-7dbf-4c19-bb55-9408d8ae1065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:07.634070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262695f-7dbf-4c19-bb55-9408d8ae1065 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:07.634416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262695f-7dbf-4c19-bb55-9408d8ae1065 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:07.634793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262695f-7dbf-4c19-bb55-9408d8ae1065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:07.646044 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262695f-7dbf-4c19-bb55-9408d8ae1065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:07.647977 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262695f-7dbf-4c19-bb55-9408d8ae1065 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:07.658294 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 870/1738] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:13.656485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0c662-0742-42cc-ae57-fb0d1a0ac81c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:13.744090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0c662-0742-42cc-ae57-fb0d1a0ac81c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:13.751189 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:13.752258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:13.752632 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:13.753204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] PATH_INFO: `/v3/users/365f391239574744be58dc5cd4ef986c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:13.755043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=365f391239574744be58dc5cd4ef986c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:13.757575 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:13.820225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365f391239574744be58dc5cd4ef986c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:13.824346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365f391239574744be58dc5cd4ef986c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:13.832883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:13.836685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365f391239574744be58dc5cd4ef986c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:13.850401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365f391239574744be58dc5cd4ef986c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:13.870445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365f391239574744be58dc5cd4ef986c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:13.879900 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 869/1739] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:49:13 2022] DELETE /identity/v3/users/365f391239574744be58dc5cd4ef986c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:13.883971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5d5ee-ec6b-41cc-a652-9c5fa21d3878 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:13.884498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5d5ee-ec6b-41cc-a652-9c5fa21d3878 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:13.884498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5d5ee-ec6b-41cc-a652-9c5fa21d3878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:13.912044 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8ed5d5ee-ec6b-41cc-a652-9c5fa21d3878 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:13.913339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8ed5d5ee-ec6b-41cc-a652-9c5fa21d3878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:13.985864 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 871/1740] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:14.000465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:14.002255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:14.118452 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89056b66-83e2-4314-a12f-31c62dcc9e18 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:14.122093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31386c63-bd35-4e44-adb1-f307b8e4de85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:14.122710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31386c63-bd35-4e44-adb1-f307b8e4de85 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:14.123010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31386c63-bd35-4e44-adb1-f307b8e4de85 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:14.123281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31386c63-bd35-4e44-adb1-f307b8e4de85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:14.134230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31386c63-bd35-4e44-adb1-f307b8e4de85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:14.135794 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31386c63-bd35-4e44-adb1-f307b8e4de85 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:14.147475 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 870/1741] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:14.516529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5d5ee-ec6b-41cc-a652-9c5fa21d3878 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:14.518094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8ed5d5ee-ec6b-41cc-a652-9c5fa21d3878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:14.540994 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5d5ee-ec6b-41cc-a652-9c5fa21d3878 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:14.546499 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:14.547191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:14.547712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:14.547995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] PATH_INFO: `/v3/projects/0001092a700c4a76bbbfd678b6df7e51` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:14.549728 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0001092a700c4a76bbbfd678b6df7e51)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:14.551764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:14.638442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0001092a700c4a76bbbfd678b6df7e51', 'request_id': 'req-32f2ab66-e4e3-4dd0-9058-80129d726cc4'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:14.653249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0001092a700c4a76bbbfd678b6df7e51', 'request_id': 'req-32f2ab66-e4e3-4dd0-9058-80129d726cc4'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:14.664815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0001092a700c4a76bbbfd678b6df7e51', 'request_id': 'req-32f2ab66-e4e3-4dd0-9058-80129d726cc4'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:14.688776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] The token cache is being invalidate because project 0001092a700c4a76bbbfd678b6df7e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:49:14.689260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:14.689999 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-32f2ab66-e4e3-4dd0-9058-80129d726cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:14.693505 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 872/1742] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:49:14 2022] DELETE /identity/v3/projects/0001092a700c4a76bbbfd678b6df7e51 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:14.698510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df25b14-c56b-4653-ac80-d762e81d84a0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:14.699071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df25b14-c56b-4653-ac80-d762e81d84a0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:14.699347 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df25b14-c56b-4653-ac80-d762e81d84a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:14.724273 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3df25b14-c56b-4653-ac80-d762e81d84a0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:14.725474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3df25b14-c56b-4653-ac80-d762e81d84a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:14.837058 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 871/1743] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:49:14.846569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc18f4-f63a-4657-8c48-737ee782197b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:14.847016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc18f4-f63a-4657-8c48-737ee782197b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:14.847384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc18f4-f63a-4657-8c48-737ee782197b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:14.906295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e9fc18f4-f63a-4657-8c48-737ee782197b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:14.908345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e9fc18f4-f63a-4657-8c48-737ee782197b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:14.918725 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 873/1744] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:14.924278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df25b14-c56b-4653-ac80-d762e81d84a0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:14.940400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df25b14-c56b-4653-ac80-d762e81d84a0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:14.945634 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e503ff6-cccc-4ff0-87c5-520b46f151f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:14.946428 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e503ff6-cccc-4ff0-87c5-520b46f151f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:14.946680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e503ff6-cccc-4ff0-87c5-520b46f151f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:14.946993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e503ff6-cccc-4ff0-87c5-520b46f151f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:14.947658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e503ff6-cccc-4ff0-87c5-520b46f151f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:49:14.947962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e503ff6-cccc-4ff0-87c5-520b46f151f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:14.949967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e503ff6-cccc-4ff0-87c5-520b46f151f3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:14.995196 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 872/1745] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:49:15.000126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc18f4-f63a-4657-8c48-737ee782197b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.005104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:15.017497 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc18f4-f63a-4657-8c48-737ee782197b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.036097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215330ab-6ce8-47f1-a301-cc0f1d663b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.037138 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215330ab-6ce8-47f1-a301-cc0f1d663b63 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.037474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215330ab-6ce8-47f1-a301-cc0f1d663b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.037816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215330ab-6ce8-47f1-a301-cc0f1d663b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.038530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215330ab-6ce8-47f1-a301-cc0f1d663b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.046916 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215330ab-6ce8-47f1-a301-cc0f1d663b63 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.098400 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 874/1746] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:49:14 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:15.102822 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e503ff6-cccc-4ff0-87c5-520b46f151f3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.118998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e503ff6-cccc-4ff0-87c5-520b46f151f3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.234589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c9ade-2284-4c5d-bd1f-94e808a44cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.235257 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c9ade-2284-4c5d-bd1f-94e808a44cfe admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.235532 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c9ade-2284-4c5d-bd1f-94e808a44cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.235800 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c9ade-2284-4c5d-bd1f-94e808a44cfe admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.236430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c9ade-2284-4c5d-bd1f-94e808a44cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.241272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c9ade-2284-4c5d-bd1f-94e808a44cfe admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.247392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:15.270143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215330ab-6ce8-47f1-a301-cc0f1d663b63 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.274809 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 873/1747] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:49:15 2022] POST /identity/v3/users => generated 303 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:15.281361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215330ab-6ce8-47f1-a301-cc0f1d663b63 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.287322 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fded804-6dc8-48d9-a537-16bb73b61761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.288325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fded804-6dc8-48d9-a537-16bb73b61761 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.288549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fded804-6dc8-48d9-a537-16bb73b61761 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.289001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fded804-6dc8-48d9-a537-16bb73b61761 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.289908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fded804-6dc8-48d9-a537-16bb73b61761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.292011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fded804-6dc8-48d9-a537-16bb73b61761 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.303892 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 875/1748] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:15.307968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c9ade-2284-4c5d-bd1f-94e808a44cfe admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.318588 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c9ade-2284-4c5d-bd1f-94e808a44cfe admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.323562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9037c7-4e07-417f-bb59-5a9e4eb8b05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.324418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9037c7-4e07-417f-bb59-5a9e4eb8b05b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.324564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9037c7-4e07-417f-bb59-5a9e4eb8b05b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.324886 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9037c7-4e07-417f-bb59-5a9e4eb8b05b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.325847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9037c7-4e07-417f-bb59-5a9e4eb8b05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.327482 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9037c7-4e07-417f-bb59-5a9e4eb8b05b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.341369 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 874/1749] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:15.345256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fded804-6dc8-48d9-a537-16bb73b61761 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.358480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fded804-6dc8-48d9-a537-16bb73b61761 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.363902 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202757b-f943-405f-8555-efd1bfc11f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.364723 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202757b-f943-405f-8555-efd1bfc11f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.367396 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202757b-f943-405f-8555-efd1bfc11f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.367779 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202757b-f943-405f-8555-efd1bfc11f01 admin admin] PATH_INFO: `/v3/projects/ea7a0a3e6b2547aeb8f380e71c8c7630/users/ff8cd12e9ece4000bfd20bf0e8ffe752/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.386410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202757b-f943-405f-8555-efd1bfc11f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7a0a3e6b2547aeb8f380e71c8c7630, user_id=ff8cd12e9ece4000bfd20bf0e8ffe752, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.390196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202757b-f943-405f-8555-efd1bfc11f01 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.419685 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 876/1750] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:15 2022] PUT /identity/v3/projects/ea7a0a3e6b2547aeb8f380e71c8c7630/users/ff8cd12e9ece4000bfd20bf0e8ffe752/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:15.430555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8a578-3deb-4779-9a08-474394a12ade None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.431079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8a578-3deb-4779-9a08-474394a12ade None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.431480 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8a578-3deb-4779-9a08-474394a12ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.484119 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8ae8a578-3deb-4779-9a08-474394a12ade None None] MFA Rules not processed for user `ff8cd12e9ece4000bfd20bf0e8ffe752`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:15.485275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8ae8a578-3deb-4779-9a08-474394a12ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:15.568633 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 875/1751] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:15 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:15.570039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202757b-f943-405f-8555-efd1bfc11f01 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.641421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202757b-f943-405f-8555-efd1bfc11f01 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.646642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356969-0934-41bc-8184-a9ab44be4dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.647425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356969-0934-41bc-8184-a9ab44be4dbd admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.647710 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356969-0934-41bc-8184-a9ab44be4dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.648127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356969-0934-41bc-8184-a9ab44be4dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.648796 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20356969-0934-41bc-8184-a9ab44be4dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.650780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20356969-0934-41bc-8184-a9ab44be4dbd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.690522 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 877/1752] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:49:15 2022] POST /identity/v3/projects => generated 403 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:15.692448 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8a578-3deb-4779-9a08-474394a12ade None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.706137 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8a578-3deb-4779-9a08-474394a12ade None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.779863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3741a-8204-49d6-a338-ce32b2cf5d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.780485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3741a-8204-49d6-a338-ce32b2cf5d13 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.780857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3741a-8204-49d6-a338-ce32b2cf5d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.781134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3741a-8204-49d6-a338-ce32b2cf5d13 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.782270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3741a-8204-49d6-a338-ce32b2cf5d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.783831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3741a-8204-49d6-a338-ce32b2cf5d13 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.807790 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 876/1753] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:49:15 2022] POST /identity/v3/users => generated 308 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:15.811927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356969-0934-41bc-8184-a9ab44be4dbd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.822904 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356969-0934-41bc-8184-a9ab44be4dbd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.828062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcde1d5-2ffb-40be-aa2a-d6e264b1169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.828735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde1d5-2ffb-40be-aa2a-d6e264b1169b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.829019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde1d5-2ffb-40be-aa2a-d6e264b1169b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.829295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde1d5-2ffb-40be-aa2a-d6e264b1169b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.829945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcde1d5-2ffb-40be-aa2a-d6e264b1169b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.831586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcde1d5-2ffb-40be-aa2a-d6e264b1169b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.846107 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 878/1754] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:49:15.847368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3741a-8204-49d6-a338-ce32b2cf5d13 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.859627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3741a-8204-49d6-a338-ce32b2cf5d13 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.864592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dad8c-1635-4dcb-a17d-f6cae6166971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.865257 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dad8c-1635-4dcb-a17d-f6cae6166971 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.865536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dad8c-1635-4dcb-a17d-f6cae6166971 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.865804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dad8c-1635-4dcb-a17d-f6cae6166971 admin admin] PATH_INFO: `/v3/projects/e978ad30342045b9a87a9c6233ee77c3/users/88882fc73f464ef4a28fb5e2815a36f0/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.881706 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dad8c-1635-4dcb-a17d-f6cae6166971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e978ad30342045b9a87a9c6233ee77c3, user_id=88882fc73f464ef4a28fb5e2815a36f0, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.884560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dad8c-1635-4dcb-a17d-f6cae6166971 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.901284 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 877/1755] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:15 2022] PUT /identity/v3/projects/e978ad30342045b9a87a9c6233ee77c3/users/88882fc73f464ef4a28fb5e2815a36f0/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:15.905534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcde1d5-2ffb-40be-aa2a-d6e264b1169b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:15.920014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcde1d5-2ffb-40be-aa2a-d6e264b1169b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:15.962504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685ce7-532f-43c6-a4ee-81886b0f5806 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.962812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685ce7-532f-43c6-a4ee-81886b0f5806 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.963083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685ce7-532f-43c6-a4ee-81886b0f5806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.983161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f6685ce7-532f-43c6-a4ee-81886b0f5806 None None] MFA Rules not processed for user `1aa023d6ce6b45efa086c40c292f4ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:15.984923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f6685ce7-532f-43c6-a4ee-81886b0f5806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:15.985264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7192738-07e9-4035-8f56-0ec466bd0342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:15.985892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7192738-07e9-4035-8f56-0ec466bd0342 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:15.986166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7192738-07e9-4035-8f56-0ec466bd0342 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:15.986436 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7192738-07e9-4035-8f56-0ec466bd0342 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:15.987113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7192738-07e9-4035-8f56-0ec466bd0342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:15.988772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7192738-07e9-4035-8f56-0ec466bd0342 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:15.999121 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 879/1756] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:16.003424 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7192738-07e9-4035-8f56-0ec466bd0342 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:16.015508 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7192738-07e9-4035-8f56-0ec466bd0342 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:16.021129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c73eb-af10-48f9-bd4d-26ad922ade2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:16.021837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c73eb-af10-48f9-bd4d-26ad922ade2a admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:16.022162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c73eb-af10-48f9-bd4d-26ad922ade2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:16.022789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c73eb-af10-48f9-bd4d-26ad922ade2a admin admin] PATH_INFO: `/v3/projects/e978ad30342045b9a87a9c6233ee77c3/users/88882fc73f464ef4a28fb5e2815a36f0/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:16.024696 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c73eb-af10-48f9-bd4d-26ad922ade2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e978ad30342045b9a87a9c6233ee77c3, user_id=88882fc73f464ef4a28fb5e2815a36f0, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:16.025191 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:16.028261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c73eb-af10-48f9-bd4d-26ad922ade2a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:16.041354 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 880/1757] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:16 2022] PUT /identity/v3/projects/e978ad30342045b9a87a9c6233ee77c3/users/88882fc73f464ef4a28fb5e2815a36f0/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:49:16.051831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b99dd7-af03-466e-ab98-7e0aadf0c748 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:16.052430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b99dd7-af03-466e-ab98-7e0aadf0c748 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:16.052710 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b99dd7-af03-466e-ab98-7e0aadf0c748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:16.063054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:16.082974 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 878/1758] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:16.093475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-51b99dd7-af03-466e-ab98-7e0aadf0c748 None None] MFA Rules not processed for user `88882fc73f464ef4a28fb5e2815a36f0`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:16.094734 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-51b99dd7-af03-466e-ab98-7e0aadf0c748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:16.183717 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 881/1759] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:49:16.196309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6685ce7-532f-43c6-a4ee-81886b0f5806 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:16.197711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f6685ce7-532f-43c6-a4ee-81886b0f5806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:16.261597 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6685ce7-532f-43c6-a4ee-81886b0f5806 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:16.328988 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d8f1f-df24-4e56-ae5e-e595ba45d0fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:16.329501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d8f1f-df24-4e56-ae5e-e595ba45d0fd service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:16.329778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d8f1f-df24-4e56-ae5e-e595ba45d0fd service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:16.330046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d8f1f-df24-4e56-ae5e-e595ba45d0fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:16.340866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d8f1f-df24-4e56-ae5e-e595ba45d0fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:16.342609 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d8f1f-df24-4e56-ae5e-e595ba45d0fd service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:16.353545 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 879/1760] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:16 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:16.522420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b99dd7-af03-466e-ab98-7e0aadf0c748 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:16.597096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b99dd7-af03-466e-ab98-7e0aadf0c748 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:16.606030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:16.606707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:16.607019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:16.608235 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] PATH_INFO: `/v3/users/ff8cd12e9ece4000bfd20bf0e8ffe752` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:16.609481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff8cd12e9ece4000bfd20bf0e8ffe752)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:16.611514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:16.664040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff8cd12e9ece4000bfd20bf0e8ffe752'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:16.672599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff8cd12e9ece4000bfd20bf0e8ffe752'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:16.673349 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff8cd12e9ece4000bfd20bf0e8ffe752'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:16.682744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff8cd12e9ece4000bfd20bf0e8ffe752'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:16.696681 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff8cd12e9ece4000bfd20bf0e8ffe752'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:16.704382 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 882/1761] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:49:16 2022] DELETE /identity/v3/users/ff8cd12e9ece4000bfd20bf0e8ffe752 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:16.709133 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654e1ec-4887-4cca-a7e7-0dfee057c2b7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:16.709658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654e1ec-4887-4cca-a7e7-0dfee057c2b7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:16.709745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654e1ec-4887-4cca-a7e7-0dfee057c2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:16.729001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f654e1ec-4887-4cca-a7e7-0dfee057c2b7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:16.730375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f654e1ec-4887-4cca-a7e7-0dfee057c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:16.798531 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 880/1762] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:16.810336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:16.811892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:16.821955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:16.878644 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e3952-a8b9-4a97-ba95-b4c64d542d6c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:16.882209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de29b4-c337-44c7-a260-7ebc65d5aea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:16.883378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de29b4-c337-44c7-a260-7ebc65d5aea5 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:16.883378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de29b4-c337-44c7-a260-7ebc65d5aea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:16.883557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de29b4-c337-44c7-a260-7ebc65d5aea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:16.900920 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de29b4-c337-44c7-a260-7ebc65d5aea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:16.902386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de29b4-c337-44c7-a260-7ebc65d5aea5 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:16.917144 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 883/1763] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:17.342939 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654e1ec-4887-4cca-a7e7-0dfee057c2b7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:17.344466 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f654e1ec-4887-4cca-a7e7-0dfee057c2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:17.365775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654e1ec-4887-4cca-a7e7-0dfee057c2b7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:17.376647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:17.377376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:17.377627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:17.377927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] PATH_INFO: `/v3/projects/ea7a0a3e6b2547aeb8f380e71c8c7630` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:17.386986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7a0a3e6b2547aeb8f380e71c8c7630)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:17.389841 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:17.431149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:17.448334 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7a0a3e6b2547aeb8f380e71c8c7630', 'request_id': 'req-dc832b0e-4a2b-4135-bede-900711c315b8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.456589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7a0a3e6b2547aeb8f380e71c8c7630', 'request_id': 'req-dc832b0e-4a2b-4135-bede-900711c315b8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.464445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7a0a3e6b2547aeb8f380e71c8c7630', 'request_id': 'req-dc832b0e-4a2b-4135-bede-900711c315b8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.480397 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] The token cache is being invalidate because project ea7a0a3e6b2547aeb8f380e71c8c7630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:49:17.480887 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.481740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.484171 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 881/1764] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:49:17 2022] DELETE /identity/v3/projects/ea7a0a3e6b2547aeb8f380e71c8c7630 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:17.487776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de29b4-c337-44c7-a260-7ebc65d5aea5 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:17.510047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de29b4-c337-44c7-a260-7ebc65d5aea5 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:17.598199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:17.599149 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:17.599439 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:17.599906 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] PATH_INFO: `/v3/users/88882fc73f464ef4a28fb5e2815a36f0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:17.601403 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88882fc73f464ef4a28fb5e2815a36f0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:17.603217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:17.667192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88882fc73f464ef4a28fb5e2815a36f0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.678350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88882fc73f464ef4a28fb5e2815a36f0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.679324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88882fc73f464ef4a28fb5e2815a36f0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.687406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88882fc73f464ef4a28fb5e2815a36f0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.703333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-eb1d041d-6e9c-469b-8af9-e099e767ad18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88882fc73f464ef4a28fb5e2815a36f0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:17.711148 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 884/1765] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:49:17 2022] DELETE /identity/v3/users/88882fc73f464ef4a28fb5e2815a36f0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:49:18.041173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:18.042815 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:18.057574 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc832b0e-4a2b-4135-bede-900711c315b8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:18.131669 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:18.132650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.132934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.133214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] PATH_INFO: `/v3/projects/e978ad30342045b9a87a9c6233ee77c3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.134401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e978ad30342045b9a87a9c6233ee77c3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:18.136439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:18.192405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e978ad30342045b9a87a9c6233ee77c3', 'request_id': 'req-720a03db-9f4e-44c7-b584-375775f7e324'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:18.198178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e978ad30342045b9a87a9c6233ee77c3', 'request_id': 'req-720a03db-9f4e-44c7-b584-375775f7e324'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:18.205229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e978ad30342045b9a87a9c6233ee77c3', 'request_id': 'req-720a03db-9f4e-44c7-b584-375775f7e324'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:18.220759 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] The token cache is being invalidate because project e978ad30342045b9a87a9c6233ee77c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:49:18.221216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:18.222009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-720a03db-9f4e-44c7-b584-375775f7e324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:18.224341 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 882/1766] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:49:18 2022] DELETE /identity/v3/projects/e978ad30342045b9a87a9c6233ee77c3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:18.229663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf0ca1-6dba-41ca-becc-757650ea965c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.230213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf0ca1-6dba-41ca-becc-757650ea965c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.230542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf0ca1-6dba-41ca-becc-757650ea965c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.256270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-05cf0ca1-6dba-41ca-becc-757650ea965c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:18.257495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-05cf0ca1-6dba-41ca-becc-757650ea965c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:18.287850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:18.327289 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 885/1767] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:18.337383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351e984-dff9-4231-bad0-8e0c5f8bc079 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.337934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351e984-dff9-4231-bad0-8e0c5f8bc079 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.338380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351e984-dff9-4231-bad0-8e0c5f8bc079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.361146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d351e984-dff9-4231-bad0-8e0c5f8bc079 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:18.362450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d351e984-dff9-4231-bad0-8e0c5f8bc079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:18.372882 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 883/1768] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:49:18.378038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf0ca1-6dba-41ca-becc-757650ea965c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:18.390600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf0ca1-6dba-41ca-becc-757650ea965c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:18.395363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dd936-2830-4738-9fdb-bfaf74c1cd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:18.396051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dd936-2830-4738-9fdb-bfaf74c1cd85 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.396336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dd936-2830-4738-9fdb-bfaf74c1cd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.396610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dd936-2830-4738-9fdb-bfaf74c1cd85 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.397299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dd936-2830-4738-9fdb-bfaf74c1cd85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:49:18.397613 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dd936-2830-4738-9fdb-bfaf74c1cd85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:18.399256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dd936-2830-4738-9fdb-bfaf74c1cd85 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:18.415374 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 886/1769] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:49:18.417170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351e984-dff9-4231-bad0-8e0c5f8bc079 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:18.428202 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351e984-dff9-4231-bad0-8e0c5f8bc079 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:18.433134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450cf15-0759-4988-8167-34c6e48b1115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:18.433753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450cf15-0759-4988-8167-34c6e48b1115 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.434048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450cf15-0759-4988-8167-34c6e48b1115 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.434297 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450cf15-0759-4988-8167-34c6e48b1115 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.439418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450cf15-0759-4988-8167-34c6e48b1115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:18.441423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450cf15-0759-4988-8167-34c6e48b1115 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:18.472195 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 884/1770] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:49:18 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:18.476603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dd936-2830-4738-9fdb-bfaf74c1cd85 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:18.487416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dd936-2830-4738-9fdb-bfaf74c1cd85 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:18.547467 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41459c9b-60cd-4477-bc03-95a5b7d57807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:18.548186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41459c9b-60cd-4477-bc03-95a5b7d57807 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.548475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41459c9b-60cd-4477-bc03-95a5b7d57807 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.548747 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41459c9b-60cd-4477-bc03-95a5b7d57807 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.549389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41459c9b-60cd-4477-bc03-95a5b7d57807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:18.551525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41459c9b-60cd-4477-bc03-95a5b7d57807 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:18.574562 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 887/1771] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:49:18 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:18.578866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450cf15-0759-4988-8167-34c6e48b1115 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:18.589527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450cf15-0759-4988-8167-34c6e48b1115 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:18.594830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275caaf-f022-4122-ac25-77feb430ecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:18.595945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275caaf-f022-4122-ac25-77feb430ecb9 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.595945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275caaf-f022-4122-ac25-77feb430ecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.596180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275caaf-f022-4122-ac25-77feb430ecb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.596691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275caaf-f022-4122-ac25-77feb430ecb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:18.598427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275caaf-f022-4122-ac25-77feb430ecb9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:18.616621 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 885/1772] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:18.618114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41459c9b-60cd-4477-bc03-95a5b7d57807 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:18.629694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41459c9b-60cd-4477-bc03-95a5b7d57807 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:18.634945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c27aa4-b9fd-4c19-ac8e-b929ad0c6343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:18.635562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c27aa4-b9fd-4c19-ac8e-b929ad0c6343 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.635837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c27aa4-b9fd-4c19-ac8e-b929ad0c6343 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.636104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c27aa4-b9fd-4c19-ac8e-b929ad0c6343 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.636773 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c27aa4-b9fd-4c19-ac8e-b929ad0c6343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:18.638371 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c27aa4-b9fd-4c19-ac8e-b929ad0c6343 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:18.649623 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 888/1773] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:18.654011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275caaf-f022-4122-ac25-77feb430ecb9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:18.666870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275caaf-f022-4122-ac25-77feb430ecb9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:18.671881 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740d9c4-3839-47e9-91fc-70d8f4b5991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:18.672686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740d9c4-3839-47e9-91fc-70d8f4b5991a admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.672996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740d9c4-3839-47e9-91fc-70d8f4b5991a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.673269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740d9c4-3839-47e9-91fc-70d8f4b5991a admin admin] PATH_INFO: `/v3/projects/36345fb8058145c480941f4c564f2071/users/a6092b4514b94101b955215d6aae0da7/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.683697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:18.692390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740d9c4-3839-47e9-91fc-70d8f4b5991a admin admin] RBAC: Authorizing `identity:create_grant(project_id=36345fb8058145c480941f4c564f2071, user_id=a6092b4514b94101b955215d6aae0da7, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:18.695852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740d9c4-3839-47e9-91fc-70d8f4b5991a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:18.710065 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 886/1774] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:18 2022] PUT /identity/v3/projects/36345fb8058145c480941f4c564f2071/users/a6092b4514b94101b955215d6aae0da7/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:18.721404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c045-976a-4ed8-9044-c121805e1b62 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.721683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c045-976a-4ed8-9044-c121805e1b62 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.721954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c045-976a-4ed8-9044-c121805e1b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.751535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2c42c045-976a-4ed8-9044-c121805e1b62 None None] MFA Rules not processed for user `a6092b4514b94101b955215d6aae0da7`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:18.752910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2c42c045-976a-4ed8-9044-c121805e1b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:18.826393 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 889/1775] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:18.831063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740d9c4-3839-47e9-91fc-70d8f4b5991a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:18.893717 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740d9c4-3839-47e9-91fc-70d8f4b5991a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:18.899185 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9480fd-228c-4a96-9755-dc3e49ea8808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:18.899965 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9480fd-228c-4a96-9755-dc3e49ea8808 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:18.900166 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9480fd-228c-4a96-9755-dc3e49ea8808 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:18.900440 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9480fd-228c-4a96-9755-dc3e49ea8808 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:18.901154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9480fd-228c-4a96-9755-dc3e49ea8808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:18.903217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9480fd-228c-4a96-9755-dc3e49ea8808 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:18.939927 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 887/1776] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:49:18 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:49:18.943165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42c045-976a-4ed8-9044-c121805e1b62 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:18.959330 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42c045-976a-4ed8-9044-c121805e1b62 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:18.980525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:19.021346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07db83-2749-4a63-9c4a-7ecd1763cdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:19.021972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07db83-2749-4a63-9c4a-7ecd1763cdae admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.022253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07db83-2749-4a63-9c4a-7ecd1763cdae admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.022524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07db83-2749-4a63-9c4a-7ecd1763cdae admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.023190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07db83-2749-4a63-9c4a-7ecd1763cdae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:19.025113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07db83-2749-4a63-9c4a-7ecd1763cdae admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:19.049586 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 890/1777] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:49:18 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:19.053971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9480fd-228c-4a96-9755-dc3e49ea8808 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:19.067216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9480fd-228c-4a96-9755-dc3e49ea8808 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:19.073167 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e5bc-223f-42f0-9418-7952e3304af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:19.073770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e5bc-223f-42f0-9418-7952e3304af1 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.074034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e5bc-223f-42f0-9418-7952e3304af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.074368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e5bc-223f-42f0-9418-7952e3304af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.076872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e5bc-223f-42f0-9418-7952e3304af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:19.079513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e5bc-223f-42f0-9418-7952e3304af1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:19.094942 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 888/1778] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:19.100215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07db83-2749-4a63-9c4a-7ecd1763cdae admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:19.101545 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b699a-3039-4584-a2fd-4c9954a0c95f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.102261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b699a-3039-4584-a2fd-4c9954a0c95f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.102631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b699a-3039-4584-a2fd-4c9954a0c95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.111208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07db83-2749-4a63-9c4a-7ecd1763cdae admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:19.116373 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc43d8f-b11d-4be0-ad51-6d380e89dc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:19.119315 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc43d8f-b11d-4be0-ad51-6d380e89dc46 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.119610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc43d8f-b11d-4be0-ad51-6d380e89dc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.119917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc43d8f-b11d-4be0-ad51-6d380e89dc46 admin admin] PATH_INFO: `/v3/projects/59f1931030bb4e958e9ea03f3a25f311/users/b7e2bbe05444432c82d08347a5d66746/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.124363 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-630b699a-3039-4584-a2fd-4c9954a0c95f None None] MFA Rules not processed for user `bee1c05fbfb94433a97928fa2eb5cd15`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:19.125619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-630b699a-3039-4584-a2fd-4c9954a0c95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:19.136949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc43d8f-b11d-4be0-ad51-6d380e89dc46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59f1931030bb4e958e9ea03f3a25f311, user_id=b7e2bbe05444432c82d08347a5d66746, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:19.139693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc43d8f-b11d-4be0-ad51-6d380e89dc46 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:19.155371 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 891/1779] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:19 2022] PUT /identity/v3/projects/59f1931030bb4e958e9ea03f3a25f311/users/b7e2bbe05444432c82d08347a5d66746/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:19.164050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc43d8f-b11d-4be0-ad51-6d380e89dc46 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:19.196329 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 889/1780] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:19.241801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc43d8f-b11d-4be0-ad51-6d380e89dc46 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:19.256302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c19c70-fe44-4b63-9afd-0014b44c9738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:19.257006 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c19c70-fe44-4b63-9afd-0014b44c9738 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.257280 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c19c70-fe44-4b63-9afd-0014b44c9738 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.257614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c19c70-fe44-4b63-9afd-0014b44c9738 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.258266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c19c70-fe44-4b63-9afd-0014b44c9738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:19.259903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c19c70-fe44-4b63-9afd-0014b44c9738 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:19.275263 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 892/1781] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:49:19.279727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b699a-3039-4584-a2fd-4c9954a0c95f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:19.298230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b699a-3039-4584-a2fd-4c9954a0c95f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:19.307131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac345d4f-abcf-433a-ac11-e30774f71dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:19.310182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac345d4f-abcf-433a-ac11-e30774f71dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.310707 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac345d4f-abcf-433a-ac11-e30774f71dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.311279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac345d4f-abcf-433a-ac11-e30774f71dc2 admin admin] PATH_INFO: `/v3/projects/59f1931030bb4e958e9ea03f3a25f311/users/b7e2bbe05444432c82d08347a5d66746/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.315181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac345d4f-abcf-433a-ac11-e30774f71dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59f1931030bb4e958e9ea03f3a25f311, user_id=b7e2bbe05444432c82d08347a5d66746, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:19.318752 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac345d4f-abcf-433a-ac11-e30774f71dc2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:19.332475 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 890/1782] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:19 2022] PUT /identity/v3/projects/59f1931030bb4e958e9ea03f3a25f311/users/b7e2bbe05444432c82d08347a5d66746/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:49:19.347115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf863142-9278-4910-b4c8-df12f485f0de None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.347115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf863142-9278-4910-b4c8-df12f485f0de None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.347115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf863142-9278-4910-b4c8-df12f485f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.376704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cf863142-9278-4910-b4c8-df12f485f0de None None] MFA Rules not processed for user `b7e2bbe05444432c82d08347a5d66746`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:19.377915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cf863142-9278-4910-b4c8-df12f485f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:19.452128 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 893/1783] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:19.459847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac345d4f-abcf-433a-ac11-e30774f71dc2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:19.461691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ac345d4f-abcf-433a-ac11-e30774f71dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:19.538542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac345d4f-abcf-433a-ac11-e30774f71dc2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:19.543046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf3fcd-ae94-47a3-8dbb-9e5d949e107f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:19.544505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf3fcd-ae94-47a3-8dbb-9e5d949e107f service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.544505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf3fcd-ae94-47a3-8dbb-9e5d949e107f service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.544990 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf3fcd-ae94-47a3-8dbb-9e5d949e107f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.555070 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:19.558026 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf3fcd-ae94-47a3-8dbb-9e5d949e107f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:19.559766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf3fcd-ae94-47a3-8dbb-9e5d949e107f service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:19.573673 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 891/1784] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:19.584586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c54ae-40eb-4c45-a1bf-b8b78535cd73 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.585092 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c54ae-40eb-4c45-a1bf-b8b78535cd73 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.585185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c54ae-40eb-4c45-a1bf-b8b78535cd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.609701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2a4c54ae-40eb-4c45-a1bf-b8b78535cd73 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:19.610971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2a4c54ae-40eb-4c45-a1bf-b8b78535cd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:19.620524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:19.687170 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 894/1785] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:49:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:19.688715 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf3fcd-ae94-47a3-8dbb-9e5d949e107f service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:19.703936 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c54ae-40eb-4c45-a1bf-b8b78535cd73 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:19.705500 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2a4c54ae-40eb-4c45-a1bf-b8b78535cd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:19.705956 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf3fcd-ae94-47a3-8dbb-9e5d949e107f service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:19.709974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361e1e6-2f14-4b3a-a5b3-0463db5e6f83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:19.710645 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361e1e6-2f14-4b3a-a5b3-0463db5e6f83 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.710963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361e1e6-2f14-4b3a-a5b3-0463db5e6f83 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.711237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361e1e6-2f14-4b3a-a5b3-0463db5e6f83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.787694 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361e1e6-2f14-4b3a-a5b3-0463db5e6f83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:19.789316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361e1e6-2f14-4b3a-a5b3-0463db5e6f83 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:19.799172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c54ae-40eb-4c45-a1bf-b8b78535cd73 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:19.803487 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 892/1786] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:19.805626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c2688-e2fd-425d-abb5-40600cc61182 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:19.806228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2688-e2fd-425d-abb5-40600cc61182 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:19.806518 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2688-e2fd-425d-abb5-40600cc61182 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:19.806804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c2688-e2fd-425d-abb5-40600cc61182 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:19.817393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c2688-e2fd-425d-abb5-40600cc61182 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:19.819190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c2688-e2fd-425d-abb5-40600cc61182 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:19.828810 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 895/1787] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:49:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 08:49:29.798121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=30 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:29.824790 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=52 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:46.926940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361e1e6-2f14-4b3a-a5b3-0463db5e6f83 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:46.928348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b361e1e6-2f14-4b3a-a5b3-0463db5e6f83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:47.046922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361e1e6-2f14-4b3a-a5b3-0463db5e6f83 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:47.052233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:47.053022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:47.053157 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:47.053433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] PATH_INFO: `/v3/users/62d26bfde62b4fe68851ddf959e4b4d5` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:47.061664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d26bfde62b4fe68851ddf959e4b4d5)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:47.063748 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:47.154481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d26bfde62b4fe68851ddf959e4b4d5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:47.156612 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d26bfde62b4fe68851ddf959e4b4d5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:47.168746 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d26bfde62b4fe68851ddf959e4b4d5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:47.184786 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d26bfde62b4fe68851ddf959e4b4d5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:47.230388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d26bfde62b4fe68851ddf959e4b4d5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:47.244475 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 893/1788] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:49:46 2022] DELETE /identity/v3/users/62d26bfde62b4fe68851ddf959e4b4d5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:47.248965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11f84e-bf05-42ca-8cd8-4b1360f3912c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:47.249845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11f84e-bf05-42ca-8cd8-4b1360f3912c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:47.251028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11f84e-bf05-42ca-8cd8-4b1360f3912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:47.289378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4d11f84e-bf05-42ca-8cd8-4b1360f3912c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:47.292143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4d11f84e-bf05-42ca-8cd8-4b1360f3912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:47.445269 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 896/1789] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:47.455949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:47.457628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:47.567126 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0152b-f76f-47e9-a402-f903650b5dd5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:47.572888 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086bfa3-df44-4b66-af4d-ce1b5180754c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:47.577567 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086bfa3-df44-4b66-af4d-ce1b5180754c service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:47.578036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086bfa3-df44-4b66-af4d-ce1b5180754c service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:47.578773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086bfa3-df44-4b66-af4d-ce1b5180754c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:47.593301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086bfa3-df44-4b66-af4d-ce1b5180754c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:47.594797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086bfa3-df44-4b66-af4d-ce1b5180754c service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:47.605409 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 894/1790] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:48.192382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11f84e-bf05-42ca-8cd8-4b1360f3912c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:48.193827 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4d11f84e-bf05-42ca-8cd8-4b1360f3912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:48.207868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11f84e-bf05-42ca-8cd8-4b1360f3912c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:48.213429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:48.214124 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.214563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.214911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] PATH_INFO: `/v3/projects/3d640d2de11941d8b5170d141fde1209` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.216062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d640d2de11941d8b5170d141fde1209)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:48.218129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:48.264142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d640d2de11941d8b5170d141fde1209', 'request_id': 'req-725c4170-db86-40f8-8e1c-643e5e2fecd8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:48.270076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d640d2de11941d8b5170d141fde1209', 'request_id': 'req-725c4170-db86-40f8-8e1c-643e5e2fecd8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:48.277147 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d640d2de11941d8b5170d141fde1209', 'request_id': 'req-725c4170-db86-40f8-8e1c-643e5e2fecd8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:48.292012 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] The token cache is being invalidate because project 3d640d2de11941d8b5170d141fde1209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:49:48.292297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:48.292912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-725c4170-db86-40f8-8e1c-643e5e2fecd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:48.294411 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 897/1791] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:49:48 2022] DELETE /identity/v3/projects/3d640d2de11941d8b5170d141fde1209 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:48.299703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe1c72-c0b8-463b-81e7-376896ee2cb8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.300204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe1c72-c0b8-463b-81e7-376896ee2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.300307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe1c72-c0b8-463b-81e7-376896ee2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.321488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-74fe1c72-c0b8-463b-81e7-376896ee2cb8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:48.322910 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-74fe1c72-c0b8-463b-81e7-376896ee2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:48.341470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:48.387811 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 895/1792] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:48.395829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067720f-c3e7-459b-90b1-6033334271fc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.396119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067720f-c3e7-459b-90b1-6033334271fc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.396395 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067720f-c3e7-459b-90b1-6033334271fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.417417 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e067720f-c3e7-459b-90b1-6033334271fc None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:48.418658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e067720f-c3e7-459b-90b1-6033334271fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:48.426864 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 898/1793] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:48.432024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe1c72-c0b8-463b-81e7-376896ee2cb8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:48.442815 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe1c72-c0b8-463b-81e7-376896ee2cb8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:48.447829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e403e4-cb79-4a64-ae72-f4e7149f4325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:48.448517 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e403e4-cb79-4a64-ae72-f4e7149f4325 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.448790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e403e4-cb79-4a64-ae72-f4e7149f4325 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.449059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e403e4-cb79-4a64-ae72-f4e7149f4325 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.449705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e403e4-cb79-4a64-ae72-f4e7149f4325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:49:48.450017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e403e4-cb79-4a64-ae72-f4e7149f4325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:48.451634 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e403e4-cb79-4a64-ae72-f4e7149f4325 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:48.463623 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 896/1794] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:49:48.468845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067720f-c3e7-459b-90b1-6033334271fc None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:48.479210 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067720f-c3e7-459b-90b1-6033334271fc None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:48.484217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039868a-bdd4-44b8-bf7d-2954472d3a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:48.484836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039868a-bdd4-44b8-bf7d-2954472d3a12 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.485116 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039868a-bdd4-44b8-bf7d-2954472d3a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.485390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039868a-bdd4-44b8-bf7d-2954472d3a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.486032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039868a-bdd4-44b8-bf7d-2954472d3a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:48.488105 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039868a-bdd4-44b8-bf7d-2954472d3a12 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:48.516266 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 899/1795] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:49:48 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:48.520948 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e403e4-cb79-4a64-ae72-f4e7149f4325 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:48.531586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e403e4-cb79-4a64-ae72-f4e7149f4325 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:48.616392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f4f8e-074a-4f09-9c3f-086d09ae5aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:48.617031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f4f8e-074a-4f09-9c3f-086d09ae5aed admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.617313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f4f8e-074a-4f09-9c3f-086d09ae5aed admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.617589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f4f8e-074a-4f09-9c3f-086d09ae5aed admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.618269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f4f8e-074a-4f09-9c3f-086d09ae5aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:48.623585 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f4f8e-074a-4f09-9c3f-086d09ae5aed admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:48.660755 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 897/1796] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:49:48 2022] POST /identity/v3/users => generated 296 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:49:48.665007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039868a-bdd4-44b8-bf7d-2954472d3a12 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:48.689127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039868a-bdd4-44b8-bf7d-2954472d3a12 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:48.694124 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9724298-32b1-4c96-bce2-fb96eded6294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:48.694772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9724298-32b1-4c96-bce2-fb96eded6294 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.695061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9724298-32b1-4c96-bce2-fb96eded6294 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.695336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9724298-32b1-4c96-bce2-fb96eded6294 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.696037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9724298-32b1-4c96-bce2-fb96eded6294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:48.697644 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9724298-32b1-4c96-bce2-fb96eded6294 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:48.708198 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 900/1797] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:48.712441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f4f8e-074a-4f09-9c3f-086d09ae5aed admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:48.722299 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f4f8e-074a-4f09-9c3f-086d09ae5aed admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:48.726945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e8774-8abf-496c-9bd6-0946dc7588b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:48.727656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8774-8abf-496c-9bd6-0946dc7588b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.727940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8774-8abf-496c-9bd6-0946dc7588b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.728217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e8774-8abf-496c-9bd6-0946dc7588b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.728880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e8774-8abf-496c-9bd6-0946dc7588b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:48.730492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e8774-8abf-496c-9bd6-0946dc7588b7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:48.742763 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 898/1798] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:49:48.747268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9724298-32b1-4c96-bce2-fb96eded6294 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:48.758591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9724298-32b1-4c96-bce2-fb96eded6294 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:48.764911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb985f-7746-434e-9549-b5361c91905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:48.765857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb985f-7746-434e-9549-b5361c91905e admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.766444 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb985f-7746-434e-9549-b5361c91905e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.766749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb985f-7746-434e-9549-b5361c91905e admin admin] PATH_INFO: `/v3/projects/b2a960bd0f1c4422bef757f5665a346f/users/ccdf374465a54eed9e477519b6ab9975/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.785828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb985f-7746-434e-9549-b5361c91905e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a960bd0f1c4422bef757f5665a346f, user_id=ccdf374465a54eed9e477519b6ab9975, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:48.789278 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb985f-7746-434e-9549-b5361c91905e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:48.816103 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 901/1799] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:48 2022] PUT /identity/v3/projects/b2a960bd0f1c4422bef757f5665a346f/users/ccdf374465a54eed9e477519b6ab9975/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:48.818006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae153-9130-4990-a0b1-609640086ffd None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:48.818641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae153-9130-4990-a0b1-609640086ffd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:48.819019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae153-9130-4990-a0b1-609640086ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:48.855930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c92ae153-9130-4990-a0b1-609640086ffd None None] MFA Rules not processed for user `ccdf374465a54eed9e477519b6ab9975`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:48.857539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c92ae153-9130-4990-a0b1-609640086ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:48.932551 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 899/1800] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:48.955349 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb985f-7746-434e-9549-b5361c91905e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:48.957101 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-85bb985f-7746-434e-9549-b5361c91905e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:48.979639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:49.025371 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb985f-7746-434e-9549-b5361c91905e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:49.029128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43304277-d8b7-4d4f-8b65-cdde8ffa2494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:49.029781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43304277-d8b7-4d4f-8b65-cdde8ffa2494 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:49.030061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43304277-d8b7-4d4f-8b65-cdde8ffa2494 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:49.030355 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43304277-d8b7-4d4f-8b65-cdde8ffa2494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:49.039384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43304277-d8b7-4d4f-8b65-cdde8ffa2494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:49.041283 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43304277-d8b7-4d4f-8b65-cdde8ffa2494 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:49.050264 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 902/1801] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:54.230900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae153-9130-4990-a0b1-609640086ffd None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:54.232713 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c92ae153-9130-4990-a0b1-609640086ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:54.273549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:54.373499 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae153-9130-4990-a0b1-609640086ffd None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:54.384979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:54.385723 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:54.386006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:54.386286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] PATH_INFO: `/v3/users/a6092b4514b94101b955215d6aae0da7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:54.387489 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6092b4514b94101b955215d6aae0da7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:54.389661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:54.446903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6092b4514b94101b955215d6aae0da7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:54.447500 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6092b4514b94101b955215d6aae0da7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:54.456097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6092b4514b94101b955215d6aae0da7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:54.478901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6092b4514b94101b955215d6aae0da7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:54.500645 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6092b4514b94101b955215d6aae0da7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:54.520036 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 900/1802] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:49:54 2022] DELETE /identity/v3/users/a6092b4514b94101b955215d6aae0da7 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:54.524543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a07ff-d28f-40a0-8d97-eb267a4cb014 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:54.525065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a07ff-d28f-40a0-8d97-eb267a4cb014 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:54.525146 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a07ff-d28f-40a0-8d97-eb267a4cb014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:54.557763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-564a07ff-d28f-40a0-8d97-eb267a4cb014 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:54.558013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-564a07ff-d28f-40a0-8d97-eb267a4cb014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:54.642146 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 903/1803] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:54.652058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:54.653465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:54.715171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525333a4-c0d3-4a80-aff2-a084459b1d65 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:54.722599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d40b32-79eb-4ccd-84dd-e6f3f14ab037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:54.723686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40b32-79eb-4ccd-84dd-e6f3f14ab037 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:54.724182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40b32-79eb-4ccd-84dd-e6f3f14ab037 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:54.724654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40b32-79eb-4ccd-84dd-e6f3f14ab037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:54.736535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d40b32-79eb-4ccd-84dd-e6f3f14ab037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:54.738010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d40b32-79eb-4ccd-84dd-e6f3f14ab037 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:54.748545 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 901/1804] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:55.105042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a07ff-d28f-40a0-8d97-eb267a4cb014 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:55.106851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-564a07ff-d28f-40a0-8d97-eb267a4cb014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:55.130785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a07ff-d28f-40a0-8d97-eb267a4cb014 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:55.138601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:55.139414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:55.139789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:55.140134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] PATH_INFO: `/v3/projects/36345fb8058145c480941f4c564f2071` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:55.141421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36345fb8058145c480941f4c564f2071)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:55.143632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:55.216585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36345fb8058145c480941f4c564f2071', 'request_id': 'req-e83721a8-c518-4dec-99fd-8359a8a458e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.223111 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36345fb8058145c480941f4c564f2071', 'request_id': 'req-e83721a8-c518-4dec-99fd-8359a8a458e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.229059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36345fb8058145c480941f4c564f2071', 'request_id': 'req-e83721a8-c518-4dec-99fd-8359a8a458e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.242313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] The token cache is being invalidate because project 36345fb8058145c480941f4c564f2071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:49:55.242610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.243463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.245146 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 904/1805] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:49:55 2022] DELETE /identity/v3/projects/36345fb8058145c480941f4c564f2071 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:55.249399 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d40b32-79eb-4ccd-84dd-e6f3f14ab037 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:55.259996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d40b32-79eb-4ccd-84dd-e6f3f14ab037 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:55.307164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:55.327607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:55.328241 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:55.328571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:55.328842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] PATH_INFO: `/v3/users/b7e2bbe05444432c82d08347a5d66746` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:55.329979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e2bbe05444432c82d08347a5d66746)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:55.331933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:55.379292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e2bbe05444432c82d08347a5d66746'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.379970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e2bbe05444432c82d08347a5d66746'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.387266 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e2bbe05444432c82d08347a5d66746'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.395644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e2bbe05444432c82d08347a5d66746'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.407654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ae142b9f-0820-4c15-9706-301040f53f08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e2bbe05444432c82d08347a5d66746'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.413608 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 902/1806] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:49:55 2022] DELETE /identity/v3/users/b7e2bbe05444432c82d08347a5d66746 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:55.692783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:55.694396 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:55.707832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83721a8-c518-4dec-99fd-8359a8a458e5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:55.751811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:55.765802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:55.766445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:55.766747 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:55.767021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] PATH_INFO: `/v3/projects/59f1931030bb4e958e9ea03f3a25f311` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:55.768205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59f1931030bb4e958e9ea03f3a25f311)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:55.770071 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:55.819844 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59f1931030bb4e958e9ea03f3a25f311', 'request_id': 'req-8df3122e-404f-48ea-854d-ca19a71e1002'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.825758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59f1931030bb4e958e9ea03f3a25f311', 'request_id': 'req-8df3122e-404f-48ea-854d-ca19a71e1002'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.831860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59f1931030bb4e958e9ea03f3a25f311', 'request_id': 'req-8df3122e-404f-48ea-854d-ca19a71e1002'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.846404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] The token cache is being invalidate because project 59f1931030bb4e958e9ea03f3a25f311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:49:55.846696 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.847425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8df3122e-404f-48ea-854d-ca19a71e1002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:55.849047 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 905/1807] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:49:55 2022] DELETE /identity/v3/projects/59f1931030bb4e958e9ea03f3a25f311 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:55.854105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae14c-5b11-46cb-8c0e-2baf88d0c0a6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:55.854502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae14c-5b11-46cb-8c0e-2baf88d0c0a6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:55.854803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae14c-5b11-46cb-8c0e-2baf88d0c0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:55.874572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-59dae14c-5b11-46cb-8c0e-2baf88d0c0a6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:55.875795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-59dae14c-5b11-46cb-8c0e-2baf88d0c0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:55.937334 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 903/1808] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:55.946255 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd54da-c3a0-4294-bb6d-93f451eebb46 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:55.946599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd54da-c3a0-4294-bb6d-93f451eebb46 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:55.947001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd54da-c3a0-4294-bb6d-93f451eebb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:55.968031 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b6fd54da-c3a0-4294-bb6d-93f451eebb46 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:55.969238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b6fd54da-c3a0-4294-bb6d-93f451eebb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:55.978162 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 906/1809] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:55.982756 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dae14c-5b11-46cb-8c0e-2baf88d0c0a6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:55.992802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dae14c-5b11-46cb-8c0e-2baf88d0c0a6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:55.998232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2f57d-3d5f-42e1-9c24-767a84e317cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:55.998864 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2f57d-3d5f-42e1-9c24-767a84e317cb admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:55.999182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2f57d-3d5f-42e1-9c24-767a84e317cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:55.999451 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2f57d-3d5f-42e1-9c24-767a84e317cb admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.000096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2f57d-3d5f-42e1-9c24-767a84e317cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:49:56.000396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2f57d-3d5f-42e1-9c24-767a84e317cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.002003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2f57d-3d5f-42e1-9c24-767a84e317cb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.014531 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 904/1810] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:49:56.019028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd54da-c3a0-4294-bb6d-93f451eebb46 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.028607 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd54da-c3a0-4294-bb6d-93f451eebb46 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.034189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e10b9-9934-4ea4-a9ba-8bb4d5efefb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.034815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e10b9-9934-4ea4-a9ba-8bb4d5efefb7 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.035091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e10b9-9934-4ea4-a9ba-8bb4d5efefb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.035361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e10b9-9934-4ea4-a9ba-8bb4d5efefb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.035986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e10b9-9934-4ea4-a9ba-8bb4d5efefb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.037887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e10b9-9934-4ea4-a9ba-8bb4d5efefb7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.070272 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 907/1811] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:49:56 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:56.074470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2f57d-3d5f-42e1-9c24-767a84e317cb admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.084381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2f57d-3d5f-42e1-9c24-767a84e317cb admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.138636 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:56.157546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca20bb-4889-4776-aad6-33b08c568278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.158301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca20bb-4889-4776-aad6-33b08c568278 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.158896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca20bb-4889-4776-aad6-33b08c568278 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.159171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca20bb-4889-4776-aad6-33b08c568278 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.159808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca20bb-4889-4776-aad6-33b08c568278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.161774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca20bb-4889-4776-aad6-33b08c568278 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.188772 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 905/1812] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:49:56 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:56.193009 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e10b9-9934-4ea4-a9ba-8bb4d5efefb7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.204365 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e10b9-9934-4ea4-a9ba-8bb4d5efefb7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.209298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c5e00-27ed-4465-b2ce-08cfceb914b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.209728 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c5e00-27ed-4465-b2ce-08cfceb914b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.210008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c5e00-27ed-4465-b2ce-08cfceb914b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.210280 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c5e00-27ed-4465-b2ce-08cfceb914b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.210969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c5e00-27ed-4465-b2ce-08cfceb914b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.212603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c5e00-27ed-4465-b2ce-08cfceb914b9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.223973 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 908/1813] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:49:56.229479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca20bb-4889-4776-aad6-33b08c568278 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.241323 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca20bb-4889-4776-aad6-33b08c568278 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.246279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3205c06-9d05-48f9-97c0-475b756937c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.246932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3205c06-9d05-48f9-97c0-475b756937c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.247211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3205c06-9d05-48f9-97c0-475b756937c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.247549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3205c06-9d05-48f9-97c0-475b756937c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.248788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3205c06-9d05-48f9-97c0-475b756937c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.250558 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3205c06-9d05-48f9-97c0-475b756937c1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.269345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c5e00-27ed-4465-b2ce-08cfceb914b9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.274968 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 906/1814] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:56.284432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c5e00-27ed-4465-b2ce-08cfceb914b9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.289716 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710c031-033b-4fec-9c4a-e9739dec4b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.290842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710c031-033b-4fec-9c4a-e9739dec4b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.291495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710c031-033b-4fec-9c4a-e9739dec4b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.291816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710c031-033b-4fec-9c4a-e9739dec4b5f admin admin] PATH_INFO: `/v3/projects/6b6535cc3f6048909507c99078ad6c5e/users/11225cbb44874dbd9ea3d00cb1e32e5f/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.311316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710c031-033b-4fec-9c4a-e9739dec4b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6535cc3f6048909507c99078ad6c5e, user_id=11225cbb44874dbd9ea3d00cb1e32e5f, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.316329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710c031-033b-4fec-9c4a-e9739dec4b5f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.333284 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 909/1815] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:56 2022] PUT /identity/v3/projects/6b6535cc3f6048909507c99078ad6c5e/users/11225cbb44874dbd9ea3d00cb1e32e5f/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:56.345318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805686c-396c-4e16-9727-4ffc14fa86d6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.345831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805686c-396c-4e16-9727-4ffc14fa86d6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.346508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805686c-396c-4e16-9727-4ffc14fa86d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.379709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d805686c-396c-4e16-9727-4ffc14fa86d6 None None] MFA Rules not processed for user `11225cbb44874dbd9ea3d00cb1e32e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:56.380934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d805686c-396c-4e16-9727-4ffc14fa86d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:56.463080 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 907/1816] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:56.464219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710c031-033b-4fec-9c4a-e9739dec4b5f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.531673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710c031-033b-4fec-9c4a-e9739dec4b5f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.536302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ccc3a-0c56-4f5c-a5c8-d5483ac0001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.536933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ccc3a-0c56-4f5c-a5c8-d5483ac0001f admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.537206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ccc3a-0c56-4f5c-a5c8-d5483ac0001f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.537486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ccc3a-0c56-4f5c-a5c8-d5483ac0001f admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.538274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ccc3a-0c56-4f5c-a5c8-d5483ac0001f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.540313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ccc3a-0c56-4f5c-a5c8-d5483ac0001f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.562683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:56.574240 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 910/1817] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:49:56 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:56.575054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805686c-396c-4e16-9727-4ffc14fa86d6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.585918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805686c-396c-4e16-9727-4ffc14fa86d6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.647079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b0c93-f149-42d3-a051-e4fa99b2ec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.647692 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b0c93-f149-42d3-a051-e4fa99b2ec38 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.647979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b0c93-f149-42d3-a051-e4fa99b2ec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.648242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b0c93-f149-42d3-a051-e4fa99b2ec38 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.648869 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b0c93-f149-42d3-a051-e4fa99b2ec38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.650813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b0c93-f149-42d3-a051-e4fa99b2ec38 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.673162 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 908/1818] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:49:56 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:49:56.677771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ccc3a-0c56-4f5c-a5c8-d5483ac0001f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.693183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ccc3a-0c56-4f5c-a5c8-d5483ac0001f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.699629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2ad3d-417f-4eb7-a1ef-dd7dcc3b7d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.700236 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2ad3d-417f-4eb7-a1ef-dd7dcc3b7d4a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.700508 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2ad3d-417f-4eb7-a1ef-dd7dcc3b7d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.700773 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2ad3d-417f-4eb7-a1ef-dd7dcc3b7d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.701416 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2ad3d-417f-4eb7-a1ef-dd7dcc3b7d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.703173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2ad3d-417f-4eb7-a1ef-dd7dcc3b7d4a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.714099 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 911/1819] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:56.718487 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b0c93-f149-42d3-a051-e4fa99b2ec38 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.728895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b0c93-f149-42d3-a051-e4fa99b2ec38 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.733921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c1f26-2c88-49e9-bf77-74db46172e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.734560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c1f26-2c88-49e9-bf77-74db46172e0b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.734910 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c1f26-2c88-49e9-bf77-74db46172e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.735187 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c1f26-2c88-49e9-bf77-74db46172e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.735822 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c1f26-2c88-49e9-bf77-74db46172e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.737487 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c1f26-2c88-49e9-bf77-74db46172e0b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.747702 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 909/1820] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:56.751931 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2ad3d-417f-4eb7-a1ef-dd7dcc3b7d4a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.762169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2ad3d-417f-4eb7-a1ef-dd7dcc3b7d4a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.767654 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2bfa2-8e0f-46fa-a2ea-57411319fef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.768294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2bfa2-8e0f-46fa-a2ea-57411319fef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.768574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2bfa2-8e0f-46fa-a2ea-57411319fef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.768854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2bfa2-8e0f-46fa-a2ea-57411319fef7 admin admin] PATH_INFO: `/v3/projects/96e43893c47241d59f3d303ee55a87ae/users/9f1bd802fdd24302a43c0511f0d3d8dd/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.784028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2bfa2-8e0f-46fa-a2ea-57411319fef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e43893c47241d59f3d303ee55a87ae, user_id=9f1bd802fdd24302a43c0511f0d3d8dd, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.786841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2bfa2-8e0f-46fa-a2ea-57411319fef7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:56.799866 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 912/1821] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:56 2022] PUT /identity/v3/projects/96e43893c47241d59f3d303ee55a87ae/users/9f1bd802fdd24302a43c0511f0d3d8dd/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:56.810493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd14f65-7a1f-4fde-8c4c-cc2f9688dede None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.810837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd14f65-7a1f-4fde-8c4c-cc2f9688dede None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.811122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd14f65-7a1f-4fde-8c4c-cc2f9688dede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.841124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4fd14f65-7a1f-4fde-8c4c-cc2f9688dede None None] MFA Rules not processed for user `9f1bd802fdd24302a43c0511f0d3d8dd`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:56.842327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4fd14f65-7a1f-4fde-8c4c-cc2f9688dede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:56.864129 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:56.910139 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 910/1822] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:56.914997 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2bfa2-8e0f-46fa-a2ea-57411319fef7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:56.976900 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2bfa2-8e0f-46fa-a2ea-57411319fef7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:56.982315 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c6072-a2ec-4566-ac28-98362e027953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:56.983023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c6072-a2ec-4566-ac28-98362e027953 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:56.983298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c6072-a2ec-4566-ac28-98362e027953 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:56.983566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c6072-a2ec-4566-ac28-98362e027953 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:56.984191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c6072-a2ec-4566-ac28-98362e027953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:56.986079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c6072-a2ec-4566-ac28-98362e027953 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:57.014941 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 913/1823] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:49:56 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:57.019083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd14f65-7a1f-4fde-8c4c-cc2f9688dede None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:57.030610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd14f65-7a1f-4fde-8c4c-cc2f9688dede None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:57.101328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f61f30-909c-48aa-aa7b-dcdbaeac4b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:57.101860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f61f30-909c-48aa-aa7b-dcdbaeac4b70 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:57.102135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f61f30-909c-48aa-aa7b-dcdbaeac4b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:57.102403 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f61f30-909c-48aa-aa7b-dcdbaeac4b70 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:57.103077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f61f30-909c-48aa-aa7b-dcdbaeac4b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:57.105406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f61f30-909c-48aa-aa7b-dcdbaeac4b70 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:57.130537 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 911/1824] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:49:57 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:49:57.134979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c6072-a2ec-4566-ac28-98362e027953 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:57.145456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c6072-a2ec-4566-ac28-98362e027953 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:57.150409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8c5f6-a09e-4648-b177-35cbfd6b7aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:57.151167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8c5f6-a09e-4648-b177-35cbfd6b7aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:57.151444 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8c5f6-a09e-4648-b177-35cbfd6b7aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:57.151711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8c5f6-a09e-4648-b177-35cbfd6b7aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:57.152361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8c5f6-a09e-4648-b177-35cbfd6b7aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:57.154022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8c5f6-a09e-4648-b177-35cbfd6b7aa8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:57.165619 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 914/1825] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:49:57.178651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f61f30-909c-48aa-aa7b-dcdbaeac4b70 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:57.188730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f61f30-909c-48aa-aa7b-dcdbaeac4b70 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:57.193285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfc828-ed3c-4c8a-8785-7af04846f50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:57.193940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfc828-ed3c-4c8a-8785-7af04846f50d admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:57.194428 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfc828-ed3c-4c8a-8785-7af04846f50d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:57.194703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfc828-ed3c-4c8a-8785-7af04846f50d admin admin] PATH_INFO: `/v3/projects/8135fe46e91e491396843eb1f689b15f/users/3eee64bced9c4d35a10a677fd0bf341a/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:57.209614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfc828-ed3c-4c8a-8785-7af04846f50d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8135fe46e91e491396843eb1f689b15f, user_id=3eee64bced9c4d35a10a677fd0bf341a, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:57.212339 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfc828-ed3c-4c8a-8785-7af04846f50d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:57.227026 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 912/1826] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:57 2022] PUT /identity/v3/projects/8135fe46e91e491396843eb1f689b15f/users/3eee64bced9c4d35a10a677fd0bf341a/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:57.231190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8c5f6-a09e-4648-b177-35cbfd6b7aa8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:57.241149 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8c5f6-a09e-4648-b177-35cbfd6b7aa8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:57.298728 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48fcb3-0a08-4873-bb80-d04f3c524f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:57.299518 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48fcb3-0a08-4873-bb80-d04f3c524f9c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:57.299796 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48fcb3-0a08-4873-bb80-d04f3c524f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:57.300060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48fcb3-0a08-4873-bb80-d04f3c524f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:57.300753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48fcb3-0a08-4873-bb80-d04f3c524f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:57.302349 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48fcb3-0a08-4873-bb80-d04f3c524f9c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:57.312704 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 915/1827] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:49:57.317177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfc828-ed3c-4c8a-8785-7af04846f50d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:57.327693 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfc828-ed3c-4c8a-8785-7af04846f50d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:57.332387 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401acb3-6dd0-40f2-a6aa-545ccb3c025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:57.333030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401acb3-6dd0-40f2-a6aa-545ccb3c025e admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:57.333308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401acb3-6dd0-40f2-a6aa-545ccb3c025e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:57.333576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401acb3-6dd0-40f2-a6aa-545ccb3c025e admin admin] PATH_INFO: `/v3/projects/8135fe46e91e491396843eb1f689b15f/users/3eee64bced9c4d35a10a677fd0bf341a/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:57.335451 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401acb3-6dd0-40f2-a6aa-545ccb3c025e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8135fe46e91e491396843eb1f689b15f, user_id=3eee64bced9c4d35a10a677fd0bf341a, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:57.338046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401acb3-6dd0-40f2-a6aa-545ccb3c025e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:57.348580 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 913/1828] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:49:57 2022] PUT /identity/v3/projects/8135fe46e91e491396843eb1f689b15f/users/3eee64bced9c4d35a10a677fd0bf341a/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:57.359846 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9702-6c91-459f-ba13-af37a7a24bc6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:57.360057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9702-6c91-459f-ba13-af37a7a24bc6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:57.360407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9702-6c91-459f-ba13-af37a7a24bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:57.397413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9e8f9702-6c91-459f-ba13-af37a7a24bc6 None None] MFA Rules not processed for user `3eee64bced9c4d35a10a677fd0bf341a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:57.398613 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9e8f9702-6c91-459f-ba13-af37a7a24bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:57.420192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:57.468913 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 916/1829] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:57.480420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401acb3-6dd0-40f2-a6aa-545ccb3c025e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:57.487588 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3401acb3-6dd0-40f2-a6aa-545ccb3c025e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:57.554457 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401acb3-6dd0-40f2-a6aa-545ccb3c025e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:57.558262 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d50d3-17d5-44cf-a0be-3d8e1d88a6e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:57.558916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d50d3-17d5-44cf-a0be-3d8e1d88a6e2 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:57.559622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d50d3-17d5-44cf-a0be-3d8e1d88a6e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:57.559622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d50d3-17d5-44cf-a0be-3d8e1d88a6e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:57.569744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d50d3-17d5-44cf-a0be-3d8e1d88a6e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:57.571215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d50d3-17d5-44cf-a0be-3d8e1d88a6e2 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:57.581552 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 914/1830] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:58.135784 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f9702-6c91-459f-ba13-af37a7a24bc6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:58.137167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9e8f9702-6c91-459f-ba13-af37a7a24bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:58.149249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f9702-6c91-459f-ba13-af37a7a24bc6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:58.152914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c84be1-d7fe-439b-b5cd-9bbfef7746d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:58.153541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c84be1-d7fe-439b-b5cd-9bbfef7746d3 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:58.153818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c84be1-d7fe-439b-b5cd-9bbfef7746d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:58.154090 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c84be1-d7fe-439b-b5cd-9bbfef7746d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:58.210023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c84be1-d7fe-439b-b5cd-9bbfef7746d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:58.211570 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c84be1-d7fe-439b-b5cd-9bbfef7746d3 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:58.220997 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 917/1831] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:49:59.619899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d50d3-17d5-44cf-a0be-3d8e1d88a6e2 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:59.621595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-615d50d3-17d5-44cf-a0be-3d8e1d88a6e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:59.644308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:49:59.696115 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d50d3-17d5-44cf-a0be-3d8e1d88a6e2 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:49:59.700901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:49:59.701407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:59.701690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:59.701965 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] PATH_INFO: `/v3/users/ccdf374465a54eed9e477519b6ab9975` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:59.703130 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccdf374465a54eed9e477519b6ab9975)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:49:59.705053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:49:59.760770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccdf374465a54eed9e477519b6ab9975'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:59.761525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccdf374465a54eed9e477519b6ab9975'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:59.769980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccdf374465a54eed9e477519b6ab9975'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:59.785030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccdf374465a54eed9e477519b6ab9975'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:59.801585 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccdf374465a54eed9e477519b6ab9975'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:49:59.810497 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 915/1832] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:49:59 2022] DELETE /identity/v3/users/ccdf374465a54eed9e477519b6ab9975 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:49:59.815174 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b11f-7383-440b-9bf5-bb7cab1e5133 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:49:59.815669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b11f-7383-440b-9bf5-bb7cab1e5133 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:49:59.815751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b11f-7383-440b-9bf5-bb7cab1e5133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:49:59.841502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5086b11f-7383-440b-9bf5-bb7cab1e5133 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:49:59.843512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5086b11f-7383-440b-9bf5-bb7cab1e5133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:49:59.927799 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 918/1833] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:49:59.939495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:49:59.941616 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:00.018237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5a75b-b8b4-4bd9-8492-af9e9bfd20a0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:00.022477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffafa6-979d-4fb8-8cc7-63e82fc2880c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:00.023180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffafa6-979d-4fb8-8cc7-63e82fc2880c service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:00.023554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffafa6-979d-4fb8-8cc7-63e82fc2880c service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:00.023871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffafa6-979d-4fb8-8cc7-63e82fc2880c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:00.033906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffafa6-979d-4fb8-8cc7-63e82fc2880c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:00.035485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffafa6-979d-4fb8-8cc7-63e82fc2880c service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:00.046333 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 916/1834] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:00.432360 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b11f-7383-440b-9bf5-bb7cab1e5133 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:00.434328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5086b11f-7383-440b-9bf5-bb7cab1e5133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:00.457981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b11f-7383-440b-9bf5-bb7cab1e5133 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:00.462896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:00.463578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:00.464070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:00.464245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] PATH_INFO: `/v3/projects/b2a960bd0f1c4422bef757f5665a346f` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:00.465426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a960bd0f1c4422bef757f5665a346f)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:00.467430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:00.513903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a960bd0f1c4422bef757f5665a346f', 'request_id': 'req-a455216f-992e-4985-8e8f-bfc7e4173aee'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:00.515796 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:00.519980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a960bd0f1c4422bef757f5665a346f', 'request_id': 'req-a455216f-992e-4985-8e8f-bfc7e4173aee'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:00.526311 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a960bd0f1c4422bef757f5665a346f', 'request_id': 'req-a455216f-992e-4985-8e8f-bfc7e4173aee'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:00.540158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] The token cache is being invalidate because project b2a960bd0f1c4422bef757f5665a346f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:00.540368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:00.541474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:00.543210 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 919/1835] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:00 2022] DELETE /identity/v3/projects/b2a960bd0f1c4422bef757f5665a346f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:02.470350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffafa6-979d-4fb8-8cc7-63e82fc2880c service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:02.471841 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-69ffafa6-979d-4fb8-8cc7-63e82fc2880c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:02.519248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:02.534191 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffafa6-979d-4fb8-8cc7-63e82fc2880c service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:02.582788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb0b04-cb77-4e0e-9589-cab18c6c8ef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:02.583401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb0b04-cb77-4e0e-9589-cab18c6c8ef3 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:02.583679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb0b04-cb77-4e0e-9589-cab18c6c8ef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:02.583948 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb0b04-cb77-4e0e-9589-cab18c6c8ef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:02.585693 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a8bb0b04-cb77-4e0e-9589-cab18c6c8ef3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:02.654213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb0b04-cb77-4e0e-9589-cab18c6c8ef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:02.655678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb0b04-cb77-4e0e-9589-cab18c6c8ef3 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:02.665012 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 917/1836] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:04.834539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:04.836026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:04.936250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455216f-992e-4985-8e8f-bfc7e4173aee admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:04.946669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:04.947428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:04.947705 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:04.947976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] PATH_INFO: `/v3/users/11225cbb44874dbd9ea3d00cb1e32e5f` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:04.949259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] RBAC: Authorizing `identity:delete_user(user_id=11225cbb44874dbd9ea3d00cb1e32e5f)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:04.951282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:05.028286 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11225cbb44874dbd9ea3d00cb1e32e5f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.040818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11225cbb44874dbd9ea3d00cb1e32e5f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.042115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11225cbb44874dbd9ea3d00cb1e32e5f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.056347 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11225cbb44874dbd9ea3d00cb1e32e5f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.073655 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11225cbb44874dbd9ea3d00cb1e32e5f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.083559 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 920/1837] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:04 2022] DELETE /identity/v3/users/11225cbb44874dbd9ea3d00cb1e32e5f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:05.086094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b04221-b9f7-4e6e-bbf0-4b8e5d4cb6f1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:05.086094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b04221-b9f7-4e6e-bbf0-4b8e5d4cb6f1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:05.086686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b04221-b9f7-4e6e-bbf0-4b8e5d4cb6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:05.111785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-33b04221-b9f7-4e6e-bbf0-4b8e5d4cb6f1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:05.113586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-33b04221-b9f7-4e6e-bbf0-4b8e5d4cb6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:05.185025 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 918/1838] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:50:05.195448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:05.197870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:05.283693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4cbf1-a7d1-4ee8-a548-4cba331cf23a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:05.288258 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec79b5-eebf-4f4d-bc80-a932cba0e9d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:05.289962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec79b5-eebf-4f4d-bc80-a932cba0e9d8 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:05.290254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec79b5-eebf-4f4d-bc80-a932cba0e9d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:05.290530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec79b5-eebf-4f4d-bc80-a932cba0e9d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:05.309840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec79b5-eebf-4f4d-bc80-a932cba0e9d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:05.311914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec79b5-eebf-4f4d-bc80-a932cba0e9d8 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:05.323556 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 921/1839] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:05.347906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b04221-b9f7-4e6e-bbf0-4b8e5d4cb6f1 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:05.349509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-33b04221-b9f7-4e6e-bbf0-4b8e5d4cb6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:05.362504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b04221-b9f7-4e6e-bbf0-4b8e5d4cb6f1 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:05.366193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672c6cc-b96e-450a-8286-77e837b2b0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:05.366913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672c6cc-b96e-450a-8286-77e837b2b0ef service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:05.367257 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672c6cc-b96e-450a-8286-77e837b2b0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:05.367600 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672c6cc-b96e-450a-8286-77e837b2b0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:05.369443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e672c6cc-b96e-450a-8286-77e837b2b0ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:05.423219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:05.440235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672c6cc-b96e-450a-8286-77e837b2b0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:05.441680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672c6cc-b96e-450a-8286-77e837b2b0ef service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:05.452092 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 919/1840] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:05.612826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec79b5-eebf-4f4d-bc80-a932cba0e9d8 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:05.614421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-77ec79b5-eebf-4f4d-bc80-a932cba0e9d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:05.623369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:05.630424 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec79b5-eebf-4f4d-bc80-a932cba0e9d8 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:05.637440 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:05.638319 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:05.638855 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:05.639650 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] PATH_INFO: `/v3/projects/6b6535cc3f6048909507c99078ad6c5e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:05.641088 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b6535cc3f6048909507c99078ad6c5e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:05.643694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:05.694179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b6535cc3f6048909507c99078ad6c5e', 'request_id': 'req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.699622 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b6535cc3f6048909507c99078ad6c5e', 'request_id': 'req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.706022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b6535cc3f6048909507c99078ad6c5e', 'request_id': 'req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.719502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] The token cache is being invalidate because project 6b6535cc3f6048909507c99078ad6c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:05.719793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.720472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.722296 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 922/1841] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:05 2022] DELETE /identity/v3/projects/6b6535cc3f6048909507c99078ad6c5e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:05.726231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672c6cc-b96e-450a-8286-77e837b2b0ef service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:05.737945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672c6cc-b96e-450a-8286-77e837b2b0ef service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:05.799014 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:05.799660 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:05.799938 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:05.800211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] PATH_INFO: `/v3/users/9f1bd802fdd24302a43c0511f0d3d8dd` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:05.801410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f1bd802fdd24302a43c0511f0d3d8dd)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:05.803376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:05.852838 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f1bd802fdd24302a43c0511f0d3d8dd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.853429 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f1bd802fdd24302a43c0511f0d3d8dd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.860603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f1bd802fdd24302a43c0511f0d3d8dd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.869564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f1bd802fdd24302a43c0511f0d3d8dd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.881886 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f1bd802fdd24302a43c0511f0d3d8dd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:05.890985 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 920/1842] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:05 2022] DELETE /identity/v3/users/9f1bd802fdd24302a43c0511f0d3d8dd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:06.219032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:06.220993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:06.235449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9b170-4f0c-4a7c-b0a9-096b62b0cec6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:06.319340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:06.319992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:06.320274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:06.320550 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] PATH_INFO: `/v3/projects/96e43893c47241d59f3d303ee55a87ae` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:06.321682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e43893c47241d59f3d303ee55a87ae)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:06.323659 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:06.372004 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e43893c47241d59f3d303ee55a87ae', 'request_id': 'req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.378202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e43893c47241d59f3d303ee55a87ae', 'request_id': 'req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.384396 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e43893c47241d59f3d303ee55a87ae', 'request_id': 'req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.397563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] The token cache is being invalidate because project 96e43893c47241d59f3d303ee55a87ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:06.397855 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.398476 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.400076 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 923/1843] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:06 2022] DELETE /identity/v3/projects/96e43893c47241d59f3d303ee55a87ae => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:06.404250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:06.405749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:06.418947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc30ee-c506-4b84-ab6f-f4a4efa8236f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:06.473980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:06.474628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:06.474940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:06.475216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] PATH_INFO: `/v3/users/3eee64bced9c4d35a10a677fd0bf341a` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:06.476344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eee64bced9c4d35a10a677fd0bf341a)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:06.478272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:06.483432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:06.528934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eee64bced9c4d35a10a677fd0bf341a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.529749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eee64bced9c4d35a10a677fd0bf341a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.537630 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eee64bced9c4d35a10a677fd0bf341a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.546889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eee64bced9c4d35a10a677fd0bf341a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.559284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-795f2df2-70b2-49b4-bf18-c8dddfbee332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eee64bced9c4d35a10a677fd0bf341a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:06.565592 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 921/1844] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:06 2022] DELETE /identity/v3/users/3eee64bced9c4d35a10a677fd0bf341a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:06.851420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:06.852960 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:06.867352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3fcab-9969-4a36-9581-b69f2a3f3dc8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:06.894542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:06.959365 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:06.960021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:06.960297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:06.960569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] PATH_INFO: `/v3/projects/8135fe46e91e491396843eb1f689b15f` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:06.961683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8135fe46e91e491396843eb1f689b15f)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:06.963684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:07.027376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8135fe46e91e491396843eb1f689b15f', 'request_id': 'req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:07.034441 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8135fe46e91e491396843eb1f689b15f', 'request_id': 'req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:07.042418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8135fe46e91e491396843eb1f689b15f', 'request_id': 'req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:07.068208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] The token cache is being invalidate because project 8135fe46e91e491396843eb1f689b15f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:07.068651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:07.069993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d018ac9-74e2-4c27-b61f-b6cb0c81de17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:07.072081 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 924/1845] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:06 2022] DELETE /identity/v3/projects/8135fe46e91e491396843eb1f689b15f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:07.079373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bff70-bb8b-4241-9fd4-29debe833465 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.079684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bff70-bb8b-4241-9fd4-29debe833465 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.080207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bff70-bb8b-4241-9fd4-29debe833465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.104703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b69bff70-bb8b-4241-9fd4-29debe833465 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:07.106334 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b69bff70-bb8b-4241-9fd4-29debe833465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:07.197348 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 922/1846] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:07.206208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67095e20-2bef-4f2e-8a31-72a02d27c3f6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.206508 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67095e20-2bef-4f2e-8a31-72a02d27c3f6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.206825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67095e20-2bef-4f2e-8a31-72a02d27c3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.229238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-67095e20-2bef-4f2e-8a31-72a02d27c3f6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:07.230506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-67095e20-2bef-4f2e-8a31-72a02d27c3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:07.239043 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 925/1847] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:07.243211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bff70-bb8b-4241-9fd4-29debe833465 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:07.253144 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bff70-bb8b-4241-9fd4-29debe833465 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:07.258805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9269c-b698-484a-aea4-dd3f208ae450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:07.259432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9269c-b698-484a-aea4-dd3f208ae450 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.259716 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9269c-b698-484a-aea4-dd3f208ae450 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.259987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9269c-b698-484a-aea4-dd3f208ae450 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.260762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9269c-b698-484a-aea4-dd3f208ae450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:50:07.260959 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9269c-b698-484a-aea4-dd3f208ae450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:07.262550 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9269c-b698-484a-aea4-dd3f208ae450 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:07.279788 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 923/1848] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:50:07.280995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67095e20-2bef-4f2e-8a31-72a02d27c3f6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:07.290227 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67095e20-2bef-4f2e-8a31-72a02d27c3f6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:07.295245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc704b-02d3-4e3a-b106-e069cdaa6a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:07.295845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc704b-02d3-4e3a-b106-e069cdaa6a8c admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.296119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc704b-02d3-4e3a-b106-e069cdaa6a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.296424 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc704b-02d3-4e3a-b106-e069cdaa6a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.297015 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc704b-02d3-4e3a-b106-e069cdaa6a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:07.298955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc704b-02d3-4e3a-b106-e069cdaa6a8c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:07.330478 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 926/1849] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:50:07 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:07.331308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9269c-b698-484a-aea4-dd3f208ae450 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:07.342230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9269c-b698-484a-aea4-dd3f208ae450 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:07.417559 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7f940-d0f7-4624-ab2c-7b440c6cdaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:07.418244 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7f940-d0f7-4624-ab2c-7b440c6cdaaa admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.418479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7f940-d0f7-4624-ab2c-7b440c6cdaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.418777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7f940-d0f7-4624-ab2c-7b440c6cdaaa admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.419436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7f940-d0f7-4624-ab2c-7b440c6cdaaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:07.422920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7f940-d0f7-4624-ab2c-7b440c6cdaaa admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:07.444908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:07.453035 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 924/1850] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:50:07 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:50:07.457793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc704b-02d3-4e3a-b106-e069cdaa6a8c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:07.472180 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc704b-02d3-4e3a-b106-e069cdaa6a8c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:07.484845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2f23-5e3e-4612-9fd8-b8c047de8189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:07.485412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2f23-5e3e-4612-9fd8-b8c047de8189 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.485803 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2f23-5e3e-4612-9fd8-b8c047de8189 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.485963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2f23-5e3e-4612-9fd8-b8c047de8189 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.486612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2f23-5e3e-4612-9fd8-b8c047de8189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:07.488357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2f23-5e3e-4612-9fd8-b8c047de8189 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:07.510004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7f940-d0f7-4624-ab2c-7b440c6cdaaa admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:07.510916 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 927/1851] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:07.524406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7f940-d0f7-4624-ab2c-7b440c6cdaaa admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:07.531599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a423c0a-0fa5-4c21-ab48-32554f97a497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:07.532542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a423c0a-0fa5-4c21-ab48-32554f97a497 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.533011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a423c0a-0fa5-4c21-ab48-32554f97a497 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.533650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a423c0a-0fa5-4c21-ab48-32554f97a497 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.534572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a423c0a-0fa5-4c21-ab48-32554f97a497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:07.536527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a423c0a-0fa5-4c21-ab48-32554f97a497 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:07.565401 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 925/1852] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:07.567101 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2f23-5e3e-4612-9fd8-b8c047de8189 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:07.578926 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2f23-5e3e-4612-9fd8-b8c047de8189 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:07.583957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74c687-2d98-4e78-bedb-ba9c706ec3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:07.584698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74c687-2d98-4e78-bedb-ba9c706ec3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.585892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74c687-2d98-4e78-bedb-ba9c706ec3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.585892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74c687-2d98-4e78-bedb-ba9c706ec3f8 admin admin] PATH_INFO: `/v3/projects/1ca3e6eeeeb44c848741d24cc83702fb/users/c4c222b5fc3b4858b2cdd7bcc1b04976/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.605020 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74c687-2d98-4e78-bedb-ba9c706ec3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca3e6eeeeb44c848741d24cc83702fb, user_id=c4c222b5fc3b4858b2cdd7bcc1b04976, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:07.608379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74c687-2d98-4e78-bedb-ba9c706ec3f8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:07.630909 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 928/1853] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:07 2022] PUT /identity/v3/projects/1ca3e6eeeeb44c848741d24cc83702fb/users/c4c222b5fc3b4858b2cdd7bcc1b04976/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:07.641063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceccb1-d971-4d69-b926-5d2091f85920 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.641296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceccb1-d971-4d69-b926-5d2091f85920 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.641577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cceccb1-d971-4d69-b926-5d2091f85920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.675226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5cceccb1-d971-4d69-b926-5d2091f85920 None None] MFA Rules not processed for user `c4c222b5fc3b4858b2cdd7bcc1b04976`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:07.676533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5cceccb1-d971-4d69-b926-5d2091f85920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:07.784187 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 926/1854] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:07.788898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74c687-2d98-4e78-bedb-ba9c706ec3f8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:07.846275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:07.865342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74c687-2d98-4e78-bedb-ba9c706ec3f8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:07.872372 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cdf4d-d469-4e20-8a07-393e77415f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:07.873027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cdf4d-d469-4e20-8a07-393e77415f74 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:07.873423 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cdf4d-d469-4e20-8a07-393e77415f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:07.874160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cdf4d-d469-4e20-8a07-393e77415f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:07.875140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cdf4d-d469-4e20-8a07-393e77415f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:07.877362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cdf4d-d469-4e20-8a07-393e77415f74 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:07.910392 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 929/1855] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:50:07 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:50:07.914906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cceccb1-d971-4d69-b926-5d2091f85920 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:07.933320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cceccb1-d971-4d69-b926-5d2091f85920 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:08.000866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73769e6a-3a10-4ce8-8aeb-d3fb3a3f2ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:08.001843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73769e6a-3a10-4ce8-8aeb-d3fb3a3f2ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:08.002251 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73769e6a-3a10-4ce8-8aeb-d3fb3a3f2ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:08.002539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73769e6a-3a10-4ce8-8aeb-d3fb3a3f2ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:08.003439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73769e6a-3a10-4ce8-8aeb-d3fb3a3f2ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:08.005569 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73769e6a-3a10-4ce8-8aeb-d3fb3a3f2ab4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:08.034285 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 927/1856] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:50:07 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:08.035085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cdf4d-d469-4e20-8a07-393e77415f74 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:08.055256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cdf4d-d469-4e20-8a07-393e77415f74 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:08.060232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ebf50-9966-408f-8f11-83c8779d4192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:08.061011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ebf50-9966-408f-8f11-83c8779d4192 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:08.061305 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ebf50-9966-408f-8f11-83c8779d4192 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:08.061682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ebf50-9966-408f-8f11-83c8779d4192 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:08.062459 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ebf50-9966-408f-8f11-83c8779d4192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:08.064179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ebf50-9966-408f-8f11-83c8779d4192 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:08.078299 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 930/1857] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:08.083121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73769e6a-3a10-4ce8-8aeb-d3fb3a3f2ab4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:08.094932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73769e6a-3a10-4ce8-8aeb-d3fb3a3f2ab4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:08.100856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ca78a-3974-4357-a337-0a2270d5c91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:08.101547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ca78a-3974-4357-a337-0a2270d5c91b admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:08.101831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ca78a-3974-4357-a337-0a2270d5c91b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:08.102416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ca78a-3974-4357-a337-0a2270d5c91b admin admin] PATH_INFO: `/v3/projects/09ac9fee09fd4d4dae031af034652097/users/12a2d446a727476a9912f3ca36166197/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:08.122196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ca78a-3974-4357-a337-0a2270d5c91b admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ac9fee09fd4d4dae031af034652097, user_id=12a2d446a727476a9912f3ca36166197, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:08.124895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ca78a-3974-4357-a337-0a2270d5c91b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:08.141764 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 928/1858] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:08 2022] PUT /identity/v3/projects/09ac9fee09fd4d4dae031af034652097/users/12a2d446a727476a9912f3ca36166197/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:08.146036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ebf50-9966-408f-8f11-83c8779d4192 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:08.156498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ebf50-9966-408f-8f11-83c8779d4192 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:08.294700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72543c7-873a-4bf1-af36-1ae8a2d5771e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:08.295546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72543c7-873a-4bf1-af36-1ae8a2d5771e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:08.295949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72543c7-873a-4bf1-af36-1ae8a2d5771e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:08.296379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72543c7-873a-4bf1-af36-1ae8a2d5771e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:08.297237 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72543c7-873a-4bf1-af36-1ae8a2d5771e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:08.299262 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72543c7-873a-4bf1-af36-1ae8a2d5771e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:08.313949 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 931/1859] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:08.317608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ca78a-3974-4357-a337-0a2270d5c91b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:08.339869 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ca78a-3974-4357-a337-0a2270d5c91b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:08.345667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cecd29-70b0-4ad3-9101-7709222ac0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:08.347050 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cecd29-70b0-4ad3-9101-7709222ac0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:08.347335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cecd29-70b0-4ad3-9101-7709222ac0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:08.347618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cecd29-70b0-4ad3-9101-7709222ac0f5 admin admin] PATH_INFO: `/v3/projects/09ac9fee09fd4d4dae031af034652097/users/12a2d446a727476a9912f3ca36166197/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:08.350526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cecd29-70b0-4ad3-9101-7709222ac0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ac9fee09fd4d4dae031af034652097, user_id=12a2d446a727476a9912f3ca36166197, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:08.353789 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cecd29-70b0-4ad3-9101-7709222ac0f5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:08.383801 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 929/1860] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:08 2022] PUT /identity/v3/projects/09ac9fee09fd4d4dae031af034652097/users/12a2d446a727476a9912f3ca36166197/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:50:08.396685 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8903b7-5a63-4c94-8e6b-9179a78a6037 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:08.397009 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8903b7-5a63-4c94-8e6b-9179a78a6037 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:08.397294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8903b7-5a63-4c94-8e6b-9179a78a6037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:08.428729 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ab8903b7-5a63-4c94-8e6b-9179a78a6037 None None] MFA Rules not processed for user `12a2d446a727476a9912f3ca36166197`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:08.429952 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ab8903b7-5a63-4c94-8e6b-9179a78a6037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:08.515055 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 932/1861] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:08.535412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cecd29-70b0-4ad3-9101-7709222ac0f5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:08.537951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b5cecd29-70b0-4ad3-9101-7709222ac0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:08.590170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:08.635171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cecd29-70b0-4ad3-9101-7709222ac0f5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:08.639465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41325e0-5034-4fef-ab69-392d91e1add6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:08.640098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41325e0-5034-4fef-ab69-392d91e1add6 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:08.640373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41325e0-5034-4fef-ab69-392d91e1add6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:08.640802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41325e0-5034-4fef-ab69-392d91e1add6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:08.651153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41325e0-5034-4fef-ab69-392d91e1add6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:08.652594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41325e0-5034-4fef-ab69-392d91e1add6 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:08.663452 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 930/1862] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:08.797285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8903b7-5a63-4c94-8e6b-9179a78a6037 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:08.820557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8903b7-5a63-4c94-8e6b-9179a78a6037 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:08.828483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1d424-b0d1-4fa0-80a2-b61be02096c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:08.829133 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1d424-b0d1-4fa0-80a2-b61be02096c1 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:08.832395 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1d424-b0d1-4fa0-80a2-b61be02096c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:08.832769 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1d424-b0d1-4fa0-80a2-b61be02096c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:08.881115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:08.909653 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1d424-b0d1-4fa0-80a2-b61be02096c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:08.911199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1d424-b0d1-4fa0-80a2-b61be02096c1 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:08.920798 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 933/1863] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:12.312996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e964d3-f7a1-4394-b50d-3ae0b491fd82 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:12.313484 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e964d3-f7a1-4394-b50d-3ae0b491fd82 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:12.313586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e964d3-f7a1-4394-b50d-3ae0b491fd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:12.334878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-22e964d3-f7a1-4394-b50d-3ae0b491fd82 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:12.336093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-22e964d3-f7a1-4394-b50d-3ae0b491fd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:12.395882 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 931/1864] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:50:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:12.422940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1d424-b0d1-4fa0-80a2-b61be02096c1 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:12.424952 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d0b1d424-b0d1-4fa0-80a2-b61be02096c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:12.497517 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1d424-b0d1-4fa0-80a2-b61be02096c1 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:12.502469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4dee-94b8-49ae-8e7f-f008a3f98018 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:12.503124 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4dee-94b8-49ae-8e7f-f008a3f98018 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:12.503405 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4dee-94b8-49ae-8e7f-f008a3f98018 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:12.503676 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4dee-94b8-49ae-8e7f-f008a3f98018 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:12.511780 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4dee-94b8-49ae-8e7f-f008a3f98018 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:12.513348 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4dee-94b8-49ae-8e7f-f008a3f98018 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:12.521620 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 934/1865] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:50:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:50:13.823359 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e964d3-f7a1-4394-b50d-3ae0b491fd82 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:13.956946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e964d3-f7a1-4394-b50d-3ae0b491fd82 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:13.961831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:13.962495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:13.962791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:13.963064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] PATH_INFO: `/v3/users/c4c222b5fc3b4858b2cdd7bcc1b04976` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:13.964382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c222b5fc3b4858b2cdd7bcc1b04976)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:13.966369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:14.028426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c222b5fc3b4858b2cdd7bcc1b04976'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.028426 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c222b5fc3b4858b2cdd7bcc1b04976'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.036398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c222b5fc3b4858b2cdd7bcc1b04976'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.047940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c222b5fc3b4858b2cdd7bcc1b04976'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.064968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c222b5fc3b4858b2cdd7bcc1b04976'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.066823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:14.073609 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 932/1866] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:13 2022] DELETE /identity/v3/users/c4c222b5fc3b4858b2cdd7bcc1b04976 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:14.078201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29cc4b-7797-4893-b551-05b7ad138b0b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:14.078849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29cc4b-7797-4893-b551-05b7ad138b0b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:14.078849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29cc4b-7797-4893-b551-05b7ad138b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:14.105027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8f29cc4b-7797-4893-b551-05b7ad138b0b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:14.106263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8f29cc4b-7797-4893-b551-05b7ad138b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:14.186547 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 935/1867] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:14.196778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:14.198203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:14.262548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e6479-6375-4e96-8087-b807b1c9d151 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:14.269797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c7cc8-122c-4784-9af5-68db81e9d181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:14.269797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c7cc8-122c-4784-9af5-68db81e9d181 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:14.269797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c7cc8-122c-4784-9af5-68db81e9d181 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:14.269797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c7cc8-122c-4784-9af5-68db81e9d181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:14.306744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c7cc8-122c-4784-9af5-68db81e9d181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:14.308962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c7cc8-122c-4784-9af5-68db81e9d181 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:14.323257 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 933/1868] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:14.674943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29cc4b-7797-4893-b551-05b7ad138b0b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:14.676460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8f29cc4b-7797-4893-b551-05b7ad138b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:14.695474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29cc4b-7797-4893-b551-05b7ad138b0b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:14.715274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:14.727794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:14.728630 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:14.729076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] PATH_INFO: `/v3/projects/1ca3e6eeeeb44c848741d24cc83702fb` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:14.731997 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca3e6eeeeb44c848741d24cc83702fb)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:14.736029 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:14.822434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca3e6eeeeb44c848741d24cc83702fb', 'request_id': 'req-d619b966-1fd3-4298-8f39-2c82218d555a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.832799 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca3e6eeeeb44c848741d24cc83702fb', 'request_id': 'req-d619b966-1fd3-4298-8f39-2c82218d555a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.835835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:14.840402 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca3e6eeeeb44c848741d24cc83702fb', 'request_id': 'req-d619b966-1fd3-4298-8f39-2c82218d555a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.859114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] The token cache is being invalidate because project 1ca3e6eeeeb44c848741d24cc83702fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:14.859469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.860025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:14.865860 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 936/1869] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:14 2022] DELETE /identity/v3/projects/1ca3e6eeeeb44c848741d24cc83702fb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:14.868357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c7cc8-122c-4784-9af5-68db81e9d181 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:14.879722 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c7cc8-122c-4784-9af5-68db81e9d181 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:14.947972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:14.949055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:14.949534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:14.949872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] PATH_INFO: `/v3/users/12a2d446a727476a9912f3ca36166197` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:14.951335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a2d446a727476a9912f3ca36166197)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:14.953582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:15.006281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a2d446a727476a9912f3ca36166197'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.007121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a2d446a727476a9912f3ca36166197'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.020427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a2d446a727476a9912f3ca36166197'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.032333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a2d446a727476a9912f3ca36166197'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.050215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc42376c-37ec-4915-9fbd-0729592ee4e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a2d446a727476a9912f3ca36166197'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.057940 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 934/1870] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:14 2022] DELETE /identity/v3/users/12a2d446a727476a9912f3ca36166197 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:15.500657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:15.502494 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:15.525119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619b966-1fd3-4298-8f39-2c82218d555a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:15.657046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:15.657715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:15.657993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:15.658291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] PATH_INFO: `/v3/projects/09ac9fee09fd4d4dae031af034652097` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:15.661255 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] RBAC: Authorizing `identity:delete_project(project_id=09ac9fee09fd4d4dae031af034652097)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:15.663341 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:15.730699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09ac9fee09fd4d4dae031af034652097', 'request_id': 'req-e0437d22-41df-461c-8a50-25caec7e845b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.738744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09ac9fee09fd4d4dae031af034652097', 'request_id': 'req-e0437d22-41df-461c-8a50-25caec7e845b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.748729 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09ac9fee09fd4d4dae031af034652097', 'request_id': 'req-e0437d22-41df-461c-8a50-25caec7e845b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.766746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] The token cache is being invalidate because project 09ac9fee09fd4d4dae031af034652097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:15.767041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.767689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e0437d22-41df-461c-8a50-25caec7e845b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:15.769944 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 937/1871] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:15 2022] DELETE /identity/v3/projects/09ac9fee09fd4d4dae031af034652097 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:15.774912 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1812fdc-234f-4bec-ad32-0b133c175fb7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:15.775201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1812fdc-234f-4bec-ad32-0b133c175fb7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:15.775474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1812fdc-234f-4bec-ad32-0b133c175fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:15.800906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f1812fdc-234f-4bec-ad32-0b133c175fb7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:15.802177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f1812fdc-234f-4bec-ad32-0b133c175fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:15.817159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:15.903933 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 935/1872] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:50:15.908373 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b75b74-8495-4432-bde9-9387e3437713 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:15.908882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b75b74-8495-4432-bde9-9387e3437713 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:15.909306 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b75b74-8495-4432-bde9-9387e3437713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:15.958680 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-33b75b74-8495-4432-bde9-9387e3437713 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:15.959956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-33b75b74-8495-4432-bde9-9387e3437713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:15.980397 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 938/1873] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:15.984852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1812fdc-234f-4bec-ad32-0b133c175fb7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:16.010955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1812fdc-234f-4bec-ad32-0b133c175fb7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:16.016494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2620f97-8130-4087-a534-e5e06adb754a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:16.017139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2620f97-8130-4087-a534-e5e06adb754a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:16.017416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2620f97-8130-4087-a534-e5e06adb754a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:16.017683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2620f97-8130-4087-a534-e5e06adb754a admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:16.018387 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2620f97-8130-4087-a534-e5e06adb754a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:50:16.018742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2620f97-8130-4087-a534-e5e06adb754a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:16.021010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2620f97-8130-4087-a534-e5e06adb754a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:16.034758 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 936/1874] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:50:16.038554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b75b74-8495-4432-bde9-9387e3437713 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:16.049031 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b75b74-8495-4432-bde9-9387e3437713 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:16.056412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbeb6c-df18-4e4a-8ac3-3dd1c4b4c039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:16.057097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbeb6c-df18-4e4a-8ac3-3dd1c4b4c039 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:16.057469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbeb6c-df18-4e4a-8ac3-3dd1c4b4c039 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:16.057847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbeb6c-df18-4e4a-8ac3-3dd1c4b4c039 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:16.058549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbeb6c-df18-4e4a-8ac3-3dd1c4b4c039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:16.060552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbeb6c-df18-4e4a-8ac3-3dd1c4b4c039 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:16.092478 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 939/1875] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:50:16 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:16.097514 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2620f97-8130-4087-a534-e5e06adb754a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:16.108725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2620f97-8130-4087-a534-e5e06adb754a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:16.167753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652f3c7-2f0c-45dc-90ce-c62f6d03802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:16.168398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652f3c7-2f0c-45dc-90ce-c62f6d03802f admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:16.168681 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652f3c7-2f0c-45dc-90ce-c62f6d03802f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:16.168960 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652f3c7-2f0c-45dc-90ce-c62f6d03802f admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:16.169595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652f3c7-2f0c-45dc-90ce-c62f6d03802f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:16.171606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652f3c7-2f0c-45dc-90ce-c62f6d03802f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:16.194267 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 937/1876] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:50:16 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:16.198471 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbeb6c-df18-4e4a-8ac3-3dd1c4b4c039 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:16.209218 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbeb6c-df18-4e4a-8ac3-3dd1c4b4c039 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:16.213854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b8c25-c31a-4086-b0e7-9ca0bf565cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:16.214463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b8c25-c31a-4086-b0e7-9ca0bf565cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:16.214766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b8c25-c31a-4086-b0e7-9ca0bf565cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:16.215046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b8c25-c31a-4086-b0e7-9ca0bf565cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:16.215682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b8c25-c31a-4086-b0e7-9ca0bf565cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:16.217283 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b8c25-c31a-4086-b0e7-9ca0bf565cd5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:16.227627 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 940/1877] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:16.231466 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652f3c7-2f0c-45dc-90ce-c62f6d03802f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:16.242593 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652f3c7-2f0c-45dc-90ce-c62f6d03802f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:16.247211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75691346-e767-4225-a5e0-2684ba82f0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:16.247814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75691346-e767-4225-a5e0-2684ba82f0af admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:16.248125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75691346-e767-4225-a5e0-2684ba82f0af admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:16.248393 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75691346-e767-4225-a5e0-2684ba82f0af admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:16.249036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75691346-e767-4225-a5e0-2684ba82f0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:16.250685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75691346-e767-4225-a5e0-2684ba82f0af admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:16.261727 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 938/1878] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:50:16.266392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b8c25-c31a-4086-b0e7-9ca0bf565cd5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:16.276578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b8c25-c31a-4086-b0e7-9ca0bf565cd5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:16.281828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2ba59-cb65-4169-bc41-852edd277dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:16.282487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2ba59-cb65-4169-bc41-852edd277dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:16.282789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2ba59-cb65-4169-bc41-852edd277dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:16.283058 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2ba59-cb65-4169-bc41-852edd277dfe admin admin] PATH_INFO: `/v3/projects/c9ae160ab65c473e8eb2604c9b4d6ab1/users/0c77eeb6a6394092af819b1fbaf28374/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:16.299529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2ba59-cb65-4169-bc41-852edd277dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ae160ab65c473e8eb2604c9b4d6ab1, user_id=0c77eeb6a6394092af819b1fbaf28374, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:16.302197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2ba59-cb65-4169-bc41-852edd277dfe admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:16.319009 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 941/1879] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:16 2022] PUT /identity/v3/projects/c9ae160ab65c473e8eb2604c9b4d6ab1/users/0c77eeb6a6394092af819b1fbaf28374/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:16.330805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0e643-10ae-4793-a5be-a6dc0f3779bd None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:16.331313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0e643-10ae-4793-a5be-a6dc0f3779bd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:16.331596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0e643-10ae-4793-a5be-a6dc0f3779bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:16.364326 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-91d0e643-10ae-4793-a5be-a6dc0f3779bd None None] MFA Rules not processed for user `0c77eeb6a6394092af819b1fbaf28374`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:16.365894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-91d0e643-10ae-4793-a5be-a6dc0f3779bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:16.442826 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 939/1880] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:50:16.455562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2ba59-cb65-4169-bc41-852edd277dfe admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:16.457434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a7d2ba59-cb65-4169-bc41-852edd277dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:16.479022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:16.535021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2ba59-cb65-4169-bc41-852edd277dfe admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:16.539007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77cd72-26e8-4a63-9a2f-03897870c9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:16.539740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77cd72-26e8-4a63-9a2f-03897870c9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:16.540150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77cd72-26e8-4a63-9a2f-03897870c9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:16.540507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77cd72-26e8-4a63-9a2f-03897870c9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:16.551556 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77cd72-26e8-4a63-9a2f-03897870c9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:16.553048 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77cd72-26e8-4a63-9a2f-03897870c9a8 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:16.565401 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 942/1881] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:18.550631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc88562-9b90-4e1a-914d-f1804e230414 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:18.551193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc88562-9b90-4e1a-914d-f1804e230414 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:18.551418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc88562-9b90-4e1a-914d-f1804e230414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:18.576871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1cc88562-9b90-4e1a-914d-f1804e230414 None None] MFA Rules not processed for user `8695364e58b04d829b12ae1b49757536`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:18.580186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1cc88562-9b90-4e1a-914d-f1804e230414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:18.596238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:18.662415 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 940/1882] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:19.892277 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77cd72-26e8-4a63-9a2f-03897870c9a8 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:20.015816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77cd72-26e8-4a63-9a2f-03897870c9a8 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:20.022497 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:20.023207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:20.023481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:20.023759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] PATH_INFO: `/v3/users/0c77eeb6a6394092af819b1fbaf28374` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:20.024940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c77eeb6a6394092af819b1fbaf28374)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:20.027060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:20.096458 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c77eeb6a6394092af819b1fbaf28374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:20.111947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c77eeb6a6394092af819b1fbaf28374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:20.112896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c77eeb6a6394092af819b1fbaf28374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:20.126032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c77eeb6a6394092af819b1fbaf28374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:20.141869 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c77eeb6a6394092af819b1fbaf28374'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:20.154410 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 943/1883] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:19 2022] DELETE /identity/v3/users/0c77eeb6a6394092af819b1fbaf28374 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:20.158580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b49097-984d-4cfe-b8b8-43576ce1e84f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:20.160317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b49097-984d-4cfe-b8b8-43576ce1e84f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:20.160627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b49097-984d-4cfe-b8b8-43576ce1e84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:20.182881 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-45b49097-984d-4cfe-b8b8-43576ce1e84f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:20.184986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-45b49097-984d-4cfe-b8b8-43576ce1e84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:20.259524 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 941/1884] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:50:20.269682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:20.271158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:20.337492 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:20.363497 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560dc1d-b941-47fa-a675-2be0e0d8618f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:20.370905 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7ec8b-0b05-4f1a-b39e-25908841812a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:20.371476 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ec8b-0b05-4f1a-b39e-25908841812a service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:20.371752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ec8b-0b05-4f1a-b39e-25908841812a service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:20.372114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ec8b-0b05-4f1a-b39e-25908841812a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:20.387779 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7ec8b-0b05-4f1a-b39e-25908841812a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:20.389352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7ec8b-0b05-4f1a-b39e-25908841812a service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:20.406931 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 944/1885] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:20.880360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b49097-984d-4cfe-b8b8-43576ce1e84f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:20.881904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-45b49097-984d-4cfe-b8b8-43576ce1e84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:20.896074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b49097-984d-4cfe-b8b8-43576ce1e84f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:20.904704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:20.905392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:20.905781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:20.906140 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] PATH_INFO: `/v3/projects/c9ae160ab65c473e8eb2604c9b4d6ab1` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:20.907493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ae160ab65c473e8eb2604c9b4d6ab1)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:20.910823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:20.976903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ae160ab65c473e8eb2604c9b4d6ab1', 'request_id': 'req-135cf777-1ce5-4c16-8c30-c7468aded960'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:20.984570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ae160ab65c473e8eb2604c9b4d6ab1', 'request_id': 'req-135cf777-1ce5-4c16-8c30-c7468aded960'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:20.992318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ae160ab65c473e8eb2604c9b4d6ab1', 'request_id': 'req-135cf777-1ce5-4c16-8c30-c7468aded960'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:21.009799 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] The token cache is being invalidate because project c9ae160ab65c473e8eb2604c9b4d6ab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:21.010088 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:21.023036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-135cf777-1ce5-4c16-8c30-c7468aded960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:21.024755 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 942/1886] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:20 2022] DELETE /identity/v3/projects/c9ae160ab65c473e8eb2604c9b4d6ab1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:21.031200 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc0228-750a-471a-abfb-df0a10423214 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.031753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc0228-750a-471a-abfb-df0a10423214 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.031843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc0228-750a-471a-abfb-df0a10423214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.070579 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-12bc0228-750a-471a-abfb-df0a10423214 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:21.071829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-12bc0228-750a-471a-abfb-df0a10423214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:21.155424 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 945/1887] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:21.163946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64806e-3764-457b-b625-c5f50102e4fb None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.164937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64806e-3764-457b-b625-c5f50102e4fb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.165546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64806e-3764-457b-b625-c5f50102e4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.195509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ba64806e-3764-457b-b625-c5f50102e4fb None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:21.197232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ba64806e-3764-457b-b625-c5f50102e4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:21.225346 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 943/1888] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:21.229461 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc0228-750a-471a-abfb-df0a10423214 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:21.245272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc0228-750a-471a-abfb-df0a10423214 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:21.251207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa08b14-15a1-43cb-87a5-02f7fe2dc65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:21.266929 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa08b14-15a1-43cb-87a5-02f7fe2dc65b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.267895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa08b14-15a1-43cb-87a5-02f7fe2dc65b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.268344 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa08b14-15a1-43cb-87a5-02f7fe2dc65b admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.269221 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa08b14-15a1-43cb-87a5-02f7fe2dc65b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:50:21.269702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa08b14-15a1-43cb-87a5-02f7fe2dc65b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:21.271879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa08b14-15a1-43cb-87a5-02f7fe2dc65b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:21.309594 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 946/1889] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:50:21.314443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64806e-3764-457b-b625-c5f50102e4fb None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:21.344639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64806e-3764-457b-b625-c5f50102e4fb None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:21.356246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d995c9e-330f-45b1-b4c4-e4911f01c30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:21.357391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d995c9e-330f-45b1-b4c4-e4911f01c30d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.357834 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d995c9e-330f-45b1-b4c4-e4911f01c30d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.358261 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d995c9e-330f-45b1-b4c4-e4911f01c30d admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.360098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d995c9e-330f-45b1-b4c4-e4911f01c30d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:21.363865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d995c9e-330f-45b1-b4c4-e4911f01c30d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:21.423157 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 944/1890] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:50:21 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:21.427632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa08b14-15a1-43cb-87a5-02f7fe2dc65b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:21.444370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa08b14-15a1-43cb-87a5-02f7fe2dc65b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:21.517778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edb2c0-6406-48b8-b71d-d233ca75028c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:21.518436 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edb2c0-6406-48b8-b71d-d233ca75028c admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.518736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edb2c0-6406-48b8-b71d-d233ca75028c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.519015 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edb2c0-6406-48b8-b71d-d233ca75028c admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.519712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edb2c0-6406-48b8-b71d-d233ca75028c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:21.521756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edb2c0-6406-48b8-b71d-d233ca75028c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:21.551532 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 947/1891] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:50:21 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:21.552891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d995c9e-330f-45b1-b4c4-e4911f01c30d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:21.564947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d995c9e-330f-45b1-b4c4-e4911f01c30d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:21.570622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74bfca-d345-4ec3-8eff-8627b072458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:21.571392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74bfca-d345-4ec3-8eff-8627b072458f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.571742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74bfca-d345-4ec3-8eff-8627b072458f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.572112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74bfca-d345-4ec3-8eff-8627b072458f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.572829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74bfca-d345-4ec3-8eff-8627b072458f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:21.574547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d74bfca-d345-4ec3-8eff-8627b072458f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:21.589505 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 945/1892] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:21.593985 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edb2c0-6406-48b8-b71d-d233ca75028c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:21.606586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edb2c0-6406-48b8-b71d-d233ca75028c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:21.612637 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a65c4-9939-40c6-b384-bf1038f77969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:21.613274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a65c4-9939-40c6-b384-bf1038f77969 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.613573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a65c4-9939-40c6-b384-bf1038f77969 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.613827 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a65c4-9939-40c6-b384-bf1038f77969 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.614496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a65c4-9939-40c6-b384-bf1038f77969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:21.616175 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a65c4-9939-40c6-b384-bf1038f77969 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:21.629692 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 948/1893] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:21.634105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74bfca-d345-4ec3-8eff-8627b072458f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:21.641194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:21.646210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74bfca-d345-4ec3-8eff-8627b072458f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:21.660142 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c8d59-f728-46ae-9334-88fd3120e2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:21.661256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c8d59-f728-46ae-9334-88fd3120e2a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.661768 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c8d59-f728-46ae-9334-88fd3120e2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.662147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c8d59-f728-46ae-9334-88fd3120e2a0 admin admin] PATH_INFO: `/v3/projects/1a287f228f574af3ad0277c7ddadeadc/users/69f0872ba1584ae090484ead6efeba6d/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.681006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c8d59-f728-46ae-9334-88fd3120e2a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a287f228f574af3ad0277c7ddadeadc, user_id=69f0872ba1584ae090484ead6efeba6d, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:21.683714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c8d59-f728-46ae-9334-88fd3120e2a0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:21.699729 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 946/1894] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:21 2022] PUT /identity/v3/projects/1a287f228f574af3ad0277c7ddadeadc/users/69f0872ba1584ae090484ead6efeba6d/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:21.710446 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71385d67-a5fd-427c-b6f5-e0fe65f32787 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.710737 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71385d67-a5fd-427c-b6f5-e0fe65f32787 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.711019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71385d67-a5fd-427c-b6f5-e0fe65f32787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.744509 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-71385d67-a5fd-427c-b6f5-e0fe65f32787 None None] MFA Rules not processed for user `69f0872ba1584ae090484ead6efeba6d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:21.745098 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-71385d67-a5fd-427c-b6f5-e0fe65f32787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:21.784481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:21.822046 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 949/1895] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:21.826402 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c8d59-f728-46ae-9334-88fd3120e2a0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:21.915657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c8d59-f728-46ae-9334-88fd3120e2a0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:21.921170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06924fd-22c6-4d61-97b0-910218eae0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:21.921836 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06924fd-22c6-4d61-97b0-910218eae0df admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:21.922119 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06924fd-22c6-4d61-97b0-910218eae0df admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:21.922392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06924fd-22c6-4d61-97b0-910218eae0df admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:21.923071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06924fd-22c6-4d61-97b0-910218eae0df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:21.924989 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06924fd-22c6-4d61-97b0-910218eae0df admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:21.965899 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 947/1896] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:50:21 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:21.967018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71385d67-a5fd-427c-b6f5-e0fe65f32787 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:21.978791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71385d67-a5fd-427c-b6f5-e0fe65f32787 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:22.055010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56c63c-7e62-4cc1-bbb9-ccecf8343776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:22.055572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56c63c-7e62-4cc1-bbb9-ccecf8343776 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:22.055774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56c63c-7e62-4cc1-bbb9-ccecf8343776 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:22.056074 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56c63c-7e62-4cc1-bbb9-ccecf8343776 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:22.058882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56c63c-7e62-4cc1-bbb9-ccecf8343776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:22.058882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56c63c-7e62-4cc1-bbb9-ccecf8343776 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:22.085223 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 950/1897] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:50:21 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:50:22.089470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06924fd-22c6-4d61-97b0-910218eae0df admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:22.102360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06924fd-22c6-4d61-97b0-910218eae0df admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:22.107159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eebb5f-9e90-4ccf-8600-225cb73774e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:22.107882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eebb5f-9e90-4ccf-8600-225cb73774e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:22.108154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eebb5f-9e90-4ccf-8600-225cb73774e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:22.108425 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eebb5f-9e90-4ccf-8600-225cb73774e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:22.109176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eebb5f-9e90-4ccf-8600-225cb73774e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:22.112767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eebb5f-9e90-4ccf-8600-225cb73774e5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:22.125672 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 948/1898] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:50:22.127014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56c63c-7e62-4cc1-bbb9-ccecf8343776 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:22.139713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56c63c-7e62-4cc1-bbb9-ccecf8343776 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:22.144881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bed2c8-aa4d-485e-8e6f-31f7c35ed310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:22.145503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed2c8-aa4d-485e-8e6f-31f7c35ed310 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:22.145781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed2c8-aa4d-485e-8e6f-31f7c35ed310 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:22.146052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed2c8-aa4d-485e-8e6f-31f7c35ed310 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:22.146699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bed2c8-aa4d-485e-8e6f-31f7c35ed310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:22.148408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bed2c8-aa4d-485e-8e6f-31f7c35ed310 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:22.159262 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 951/1899] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:22.163720 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eebb5f-9e90-4ccf-8600-225cb73774e5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:22.177754 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eebb5f-9e90-4ccf-8600-225cb73774e5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:22.183382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924310c4-5688-49e9-a9ed-84cca5293852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:22.184427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924310c4-5688-49e9-a9ed-84cca5293852 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:22.184711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924310c4-5688-49e9-a9ed-84cca5293852 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:22.184986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924310c4-5688-49e9-a9ed-84cca5293852 admin admin] PATH_INFO: `/v3/projects/c7b28c8ba89c481da35590d5b45ab2b7/users/c433fcebe63a4ffb8ccfc4b5f7820017/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:22.205451 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924310c4-5688-49e9-a9ed-84cca5293852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b28c8ba89c481da35590d5b45ab2b7, user_id=c433fcebe63a4ffb8ccfc4b5f7820017, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:22.208168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924310c4-5688-49e9-a9ed-84cca5293852 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:22.234850 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 949/1900] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:22 2022] PUT /identity/v3/projects/c7b28c8ba89c481da35590d5b45ab2b7/users/c433fcebe63a4ffb8ccfc4b5f7820017/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:50:22.309899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d6dad-e034-4060-bf7d-cfb7ac788c3d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:22.310320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d6dad-e034-4060-bf7d-cfb7ac788c3d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:22.310558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d6dad-e034-4060-bf7d-cfb7ac788c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:22.347352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-979d6dad-e034-4060-bf7d-cfb7ac788c3d None None] MFA Rules not processed for user `c433fcebe63a4ffb8ccfc4b5f7820017`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:22.348669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-979d6dad-e034-4060-bf7d-cfb7ac788c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:22.450859 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 952/1901] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:22.454695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924310c4-5688-49e9-a9ed-84cca5293852 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:22.580113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924310c4-5688-49e9-a9ed-84cca5293852 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:22.586650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a19f6f-bd4a-4e6c-a1e5-b5dfb85bca45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:22.587331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a19f6f-bd4a-4e6c-a1e5-b5dfb85bca45 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:22.587607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a19f6f-bd4a-4e6c-a1e5-b5dfb85bca45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:22.587875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a19f6f-bd4a-4e6c-a1e5-b5dfb85bca45 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:22.588532 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a19f6f-bd4a-4e6c-a1e5-b5dfb85bca45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:22.590544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a19f6f-bd4a-4e6c-a1e5-b5dfb85bca45 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:22.633294 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 950/1902] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:50:22 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:22.635240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d6dad-e034-4060-bf7d-cfb7ac788c3d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:22.652841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d6dad-e034-4060-bf7d-cfb7ac788c3d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:22.720125 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:22.767843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97897caa-0160-424e-964b-85ddc9e123ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:22.768961 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97897caa-0160-424e-964b-85ddc9e123ea admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:22.769390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97897caa-0160-424e-964b-85ddc9e123ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:22.769954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97897caa-0160-424e-964b-85ddc9e123ea admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:22.772456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97897caa-0160-424e-964b-85ddc9e123ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:22.777687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97897caa-0160-424e-964b-85ddc9e123ea admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:22.813564 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 953/1903] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:50:22 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:22.815543 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a19f6f-bd4a-4e6c-a1e5-b5dfb85bca45 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:22.840383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a19f6f-bd4a-4e6c-a1e5-b5dfb85bca45 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:22.845590 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3122bf-62f3-4cb9-8c04-4313bed5c5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:22.849992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3122bf-62f3-4cb9-8c04-4313bed5c5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:22.850386 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3122bf-62f3-4cb9-8c04-4313bed5c5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:22.850774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3122bf-62f3-4cb9-8c04-4313bed5c5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:22.851773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3122bf-62f3-4cb9-8c04-4313bed5c5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:22.853699 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3122bf-62f3-4cb9-8c04-4313bed5c5e9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:22.884072 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 951/1904] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:22.893228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97897caa-0160-424e-964b-85ddc9e123ea admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:22.931039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97897caa-0160-424e-964b-85ddc9e123ea admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:22.934291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3f044-29f7-4bd7-a0ef-67286297829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:22.935162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3f044-29f7-4bd7-a0ef-67286297829a admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:22.935372 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3f044-29f7-4bd7-a0ef-67286297829a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:22.935635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3f044-29f7-4bd7-a0ef-67286297829a admin admin] PATH_INFO: `/v3/projects/0d0b878863364a8c989602a5f620f994/users/80571ab00a1a4e76855f36e7161516a5/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:22.988521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3f044-29f7-4bd7-a0ef-67286297829a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0b878863364a8c989602a5f620f994, user_id=80571ab00a1a4e76855f36e7161516a5, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:22.991336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3f044-29f7-4bd7-a0ef-67286297829a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:23.024184 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 954/1905] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:22 2022] PUT /identity/v3/projects/0d0b878863364a8c989602a5f620f994/users/80571ab00a1a4e76855f36e7161516a5/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:23.049195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3122bf-62f3-4cb9-8c04-4313bed5c5e9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:23.074078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:23.090763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3122bf-62f3-4cb9-8c04-4313bed5c5e9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:23.183145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d9911-59d2-4669-9776-e3f812cc0884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:23.183145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d9911-59d2-4669-9776-e3f812cc0884 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:23.183145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d9911-59d2-4669-9776-e3f812cc0884 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:23.183145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d9911-59d2-4669-9776-e3f812cc0884 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:23.184030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d9911-59d2-4669-9776-e3f812cc0884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:23.195920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d9911-59d2-4669-9776-e3f812cc0884 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:23.208512 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 952/1906] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:23.213268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3f044-29f7-4bd7-a0ef-67286297829a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:23.244161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3f044-29f7-4bd7-a0ef-67286297829a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:23.251544 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e9528-d643-4a0d-9ffa-3d4be396526e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:23.253156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e9528-d643-4a0d-9ffa-3d4be396526e admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:23.253762 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e9528-d643-4a0d-9ffa-3d4be396526e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:23.255266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e9528-d643-4a0d-9ffa-3d4be396526e admin admin] PATH_INFO: `/v3/projects/0d0b878863364a8c989602a5f620f994/users/80571ab00a1a4e76855f36e7161516a5/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:23.258588 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e9528-d643-4a0d-9ffa-3d4be396526e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0b878863364a8c989602a5f620f994, user_id=80571ab00a1a4e76855f36e7161516a5, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:23.262150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e9528-d643-4a0d-9ffa-3d4be396526e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:23.280113 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 955/1907] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:23 2022] PUT /identity/v3/projects/0d0b878863364a8c989602a5f620f994/users/80571ab00a1a4e76855f36e7161516a5/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:23.290031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291717b-e964-4cc9-9c57-300a95dfe87a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:23.290685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291717b-e964-4cc9-9c57-300a95dfe87a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:23.291311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291717b-e964-4cc9-9c57-300a95dfe87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:23.334204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d291717b-e964-4cc9-9c57-300a95dfe87a None None] MFA Rules not processed for user `80571ab00a1a4e76855f36e7161516a5`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:23.336176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d291717b-e964-4cc9-9c57-300a95dfe87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:23.470475 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 953/1908] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:50:23.487704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e9528-d643-4a0d-9ffa-3d4be396526e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:23.489703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ca4e9528-d643-4a0d-9ffa-3d4be396526e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:23.582812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e9528-d643-4a0d-9ffa-3d4be396526e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:23.587123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc201b-bfa7-47a7-8366-a35554e340f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:23.588036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc201b-bfa7-47a7-8366-a35554e340f3 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:23.588514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc201b-bfa7-47a7-8366-a35554e340f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:23.588972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc201b-bfa7-47a7-8366-a35554e340f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:23.669957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc201b-bfa7-47a7-8366-a35554e340f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:23.672177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc201b-bfa7-47a7-8366-a35554e340f3 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:23.683856 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 956/1909] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:50:25.443705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291717b-e964-4cc9-9c57-300a95dfe87a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:25.447435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d291717b-e964-4cc9-9c57-300a95dfe87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:25.461394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291717b-e964-4cc9-9c57-300a95dfe87a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:25.465586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66db8f-7ea0-4d44-bf42-5e0827b8b161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:25.466395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66db8f-7ea0-4d44-bf42-5e0827b8b161 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:25.466625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66db8f-7ea0-4d44-bf42-5e0827b8b161 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:25.466772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66db8f-7ea0-4d44-bf42-5e0827b8b161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:25.534464 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66db8f-7ea0-4d44-bf42-5e0827b8b161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:25.535913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66db8f-7ea0-4d44-bf42-5e0827b8b161 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:25.546607 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 954/1910] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:26.281294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc201b-bfa7-47a7-8366-a35554e340f3 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:26.292044 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc201b-bfa7-47a7-8366-a35554e340f3 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:26.296155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04f427-1ccb-40cd-aae8-72abbfbe1a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:26.296847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04f427-1ccb-40cd-aae8-72abbfbe1a15 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:26.297124 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04f427-1ccb-40cd-aae8-72abbfbe1a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:26.297404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04f427-1ccb-40cd-aae8-72abbfbe1a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:26.307930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04f427-1ccb-40cd-aae8-72abbfbe1a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:26.309192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04f427-1ccb-40cd-aae8-72abbfbe1a15 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:26.324000 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 957/1911] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:30.431979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66db8f-7ea0-4d44-bf42-5e0827b8b161 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:30.474537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:30.523363 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66db8f-7ea0-4d44-bf42-5e0827b8b161 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:30.528914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:30.529430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:30.529714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:30.529986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] PATH_INFO: `/v3/users/69f0872ba1584ae090484ead6efeba6d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:30.531242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f0872ba1584ae090484ead6efeba6d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:30.534302 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:30.611783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f0872ba1584ae090484ead6efeba6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:30.614974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f0872ba1584ae090484ead6efeba6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:30.623573 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f0872ba1584ae090484ead6efeba6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:30.639082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f0872ba1584ae090484ead6efeba6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:30.660589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f0872ba1584ae090484ead6efeba6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:30.671597 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 955/1912] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:30 2022] DELETE /identity/v3/users/69f0872ba1584ae090484ead6efeba6d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:30.675340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee643ef-5383-4262-8208-7f0433e83d96 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:30.675891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee643ef-5383-4262-8208-7f0433e83d96 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:30.676117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee643ef-5383-4262-8208-7f0433e83d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:30.696259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-dee643ef-5383-4262-8208-7f0433e83d96 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:30.697703 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dee643ef-5383-4262-8208-7f0433e83d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:30.737399 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:30.767766 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 958/1913] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:30.782003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:30.783982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:30.856546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922cc24-d5e6-4cf0-ae6f-eb2cea2798a4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:30.865461 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292da819-9fe6-4884-accf-40860162b097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:30.866357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292da819-9fe6-4884-accf-40860162b097 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:30.866807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292da819-9fe6-4884-accf-40860162b097 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:30.867349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292da819-9fe6-4884-accf-40860162b097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:30.880308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292da819-9fe6-4884-accf-40860162b097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:30.881869 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292da819-9fe6-4884-accf-40860162b097 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:30.892389 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 956/1914] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:31.237278 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee643ef-5383-4262-8208-7f0433e83d96 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:31.238874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dee643ef-5383-4262-8208-7f0433e83d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:31.257727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee643ef-5383-4262-8208-7f0433e83d96 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:31.267652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:31.267652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:31.267652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:31.267652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] PATH_INFO: `/v3/projects/1a287f228f574af3ad0277c7ddadeadc` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:31.269370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a287f228f574af3ad0277c7ddadeadc)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:31.271460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:31.343534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a287f228f574af3ad0277c7ddadeadc', 'request_id': 'req-992369fd-560b-45ec-a9d8-6322900ab809'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.349832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a287f228f574af3ad0277c7ddadeadc', 'request_id': 'req-992369fd-560b-45ec-a9d8-6322900ab809'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.357574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a287f228f574af3ad0277c7ddadeadc', 'request_id': 'req-992369fd-560b-45ec-a9d8-6322900ab809'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.375356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] The token cache is being invalidate because project 1a287f228f574af3ad0277c7ddadeadc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:31.375765 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.376432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.378063 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 959/1915] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:31 2022] DELETE /identity/v3/projects/1a287f228f574af3ad0277c7ddadeadc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:31.382244 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292da819-9fe6-4884-accf-40860162b097 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:31.393082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292da819-9fe6-4884-accf-40860162b097 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:31.442138 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:31.456318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:31.457063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:31.457548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:31.457871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] PATH_INFO: `/v3/users/c433fcebe63a4ffb8ccfc4b5f7820017` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:31.459324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c433fcebe63a4ffb8ccfc4b5f7820017)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:31.461565 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:31.509381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c433fcebe63a4ffb8ccfc4b5f7820017'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.510073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c433fcebe63a4ffb8ccfc4b5f7820017'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.517610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c433fcebe63a4ffb8ccfc4b5f7820017'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.526421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c433fcebe63a4ffb8ccfc4b5f7820017'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.539075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c433fcebe63a4ffb8ccfc4b5f7820017'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.545564 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 957/1916] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:31 2022] DELETE /identity/v3/users/c433fcebe63a4ffb8ccfc4b5f7820017 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:31.830183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:31.831832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:31.846327 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992369fd-560b-45ec-a9d8-6322900ab809 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:31.908202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:31.908779 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:31.909047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:31.909321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] PATH_INFO: `/v3/projects/c7b28c8ba89c481da35590d5b45ab2b7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:31.910493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b28c8ba89c481da35590d5b45ab2b7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:31.912449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:31.961628 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b28c8ba89c481da35590d5b45ab2b7', 'request_id': 'req-80cdef51-f95f-4b2a-b61e-1bf2512f7104'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.967668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b28c8ba89c481da35590d5b45ab2b7', 'request_id': 'req-80cdef51-f95f-4b2a-b61e-1bf2512f7104'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.974544 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b28c8ba89c481da35590d5b45ab2b7', 'request_id': 'req-80cdef51-f95f-4b2a-b61e-1bf2512f7104'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.976395 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:31.991380 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] The token cache is being invalidate because project c7b28c8ba89c481da35590d5b45ab2b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:31.991668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.992487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-80cdef51-f95f-4b2a-b61e-1bf2512f7104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:31.994110 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 960/1917] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:31 2022] DELETE /identity/v3/projects/c7b28c8ba89c481da35590d5b45ab2b7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:31.998010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:31.999509 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:32.021099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c5db1-9bf2-4a2e-bbbd-2e562267b143 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:32.079853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35867f-a888-4e86-a4c5-fc6234b80a34 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:32.080199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35867f-a888-4e86-a4c5-fc6234b80a34 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:32.080564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35867f-a888-4e86-a4c5-fc6234b80a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:32.084316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:32.084959 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:32.085269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:32.085545 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] PATH_INFO: `/v3/users/80571ab00a1a4e76855f36e7161516a5` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:32.086799 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80571ab00a1a4e76855f36e7161516a5)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:32.088714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:32.102220 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4e35867f-a888-4e86-a4c5-fc6234b80a34 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:32.103577 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4e35867f-a888-4e86-a4c5-fc6234b80a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:32.137863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80571ab00a1a4e76855f36e7161516a5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.138515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80571ab00a1a4e76855f36e7161516a5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.146267 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80571ab00a1a4e76855f36e7161516a5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.154840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80571ab00a1a4e76855f36e7161516a5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.159567 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:32.164036 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 961/1918] 10.222.0.26 () {58 vars in 967 bytes} [Mon Oct 24 08:50:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:50:32.170376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80571ab00a1a4e76855f36e7161516a5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.177370 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 958/1919] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:31 2022] DELETE /identity/v3/users/80571ab00a1a4e76855f36e7161516a5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:32.178685 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35867f-a888-4e86-a4c5-fc6234b80a34 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:32.180269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4e35867f-a888-4e86-a4c5-fc6234b80a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:32.264053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35867f-a888-4e86-a4c5-fc6234b80a34 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:32.373480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3554e-fb66-474c-91c6-9d608201a0f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:32.374393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3554e-fb66-474c-91c6-9d608201a0f0 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:32.374626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3554e-fb66-474c-91c6-9d608201a0f0 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:32.374967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3554e-fb66-474c-91c6-9d608201a0f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:32.381079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-26f3554e-fb66-474c-91c6-9d608201a0f0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:32.435243 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:32.462069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3554e-fb66-474c-91c6-9d608201a0f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:32.464114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3554e-fb66-474c-91c6-9d608201a0f0 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:32.474737 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 962/1920] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:50:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:50:32.604787 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:32.606508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:32.744267 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1af24-8c16-4417-ad72-53ccc0a66d70 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:32.749843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:32.750567 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:32.750942 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:32.751403 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] PATH_INFO: `/v3/projects/0d0b878863364a8c989602a5f620f994` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:32.753342 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0b878863364a8c989602a5f620f994)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:32.756387 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:32.829149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0b878863364a8c989602a5f620f994', 'request_id': 'req-db0543c1-b269-45a7-b195-b6adf64ce69e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.836804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0b878863364a8c989602a5f620f994', 'request_id': 'req-db0543c1-b269-45a7-b195-b6adf64ce69e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.844367 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0b878863364a8c989602a5f620f994', 'request_id': 'req-db0543c1-b269-45a7-b195-b6adf64ce69e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.862494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] The token cache is being invalidate because project 0d0b878863364a8c989602a5f620f994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:32.862992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.863674 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-db0543c1-b269-45a7-b195-b6adf64ce69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:32.865362 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 959/1921] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:32 2022] DELETE /identity/v3/projects/0d0b878863364a8c989602a5f620f994 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:32.871474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18f76e-60c1-4bfb-b416-ecea6e89e0ae None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:32.872585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18f76e-60c1-4bfb-b416-ecea6e89e0ae None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:32.873095 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18f76e-60c1-4bfb-b416-ecea6e89e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:32.898340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2b18f76e-60c1-4bfb-b416-ecea6e89e0ae None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:32.899767 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2b18f76e-60c1-4bfb-b416-ecea6e89e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:32.998140 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 963/1922] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:33.000389 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce2278-3e50-4375-831b-230224921ee1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:33.000881 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce2278-3e50-4375-831b-230224921ee1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:33.001250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce2278-3e50-4375-831b-230224921ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:33.023840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-60ce2278-3e50-4375-831b-230224921ee1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:33.025178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-60ce2278-3e50-4375-831b-230224921ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:33.034617 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 960/1923] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:33.039451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18f76e-60c1-4bfb-b416-ecea6e89e0ae None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:33.051407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18f76e-60c1-4bfb-b416-ecea6e89e0ae None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:33.056192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ca99b-1e46-4fc3-9651-6306d10b103e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:33.056870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ca99b-1e46-4fc3-9651-6306d10b103e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:33.057084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ca99b-1e46-4fc3-9651-6306d10b103e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:33.057351 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ca99b-1e46-4fc3-9651-6306d10b103e admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:33.057997 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ca99b-1e46-4fc3-9651-6306d10b103e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:50:33.058317 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ca99b-1e46-4fc3-9651-6306d10b103e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:33.059951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ca99b-1e46-4fc3-9651-6306d10b103e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:33.072104 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 964/1924] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:50:33.076737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce2278-3e50-4375-831b-230224921ee1 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:33.086878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce2278-3e50-4375-831b-230224921ee1 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:33.091877 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6ad3-8a5d-4fd1-87d2-1266f2f797b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:33.092617 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6ad3-8a5d-4fd1-87d2-1266f2f797b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:33.092955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6ad3-8a5d-4fd1-87d2-1266f2f797b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:33.093355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6ad3-8a5d-4fd1-87d2-1266f2f797b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:33.094076 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc6ad3-8a5d-4fd1-87d2-1266f2f797b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:33.096073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc6ad3-8a5d-4fd1-87d2-1266f2f797b8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:33.124688 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 961/1925] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:50:33 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:50:33.129217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ca99b-1e46-4fc3-9651-6306d10b103e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:33.139386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ca99b-1e46-4fc3-9651-6306d10b103e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:33.196011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab6127-e2df-486e-b991-1cd3d65632ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:33.196562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab6127-e2df-486e-b991-1cd3d65632ea admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:33.196837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab6127-e2df-486e-b991-1cd3d65632ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:33.197107 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab6127-e2df-486e-b991-1cd3d65632ea admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:33.197743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab6127-e2df-486e-b991-1cd3d65632ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:33.199706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab6127-e2df-486e-b991-1cd3d65632ea admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:33.221827 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 965/1926] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:50:33 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:33.226555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6ad3-8a5d-4fd1-87d2-1266f2f797b8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:33.237699 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6ad3-8a5d-4fd1-87d2-1266f2f797b8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:33.243170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bff70-fd92-4b0c-9abb-803eb863c870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:33.243768 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bff70-fd92-4b0c-9abb-803eb863c870 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:33.244037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bff70-fd92-4b0c-9abb-803eb863c870 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:33.244308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bff70-fd92-4b0c-9abb-803eb863c870 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:33.244950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bff70-fd92-4b0c-9abb-803eb863c870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:33.246612 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bff70-fd92-4b0c-9abb-803eb863c870 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:33.258506 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 962/1927] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:33.261426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab6127-e2df-486e-b991-1cd3d65632ea admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:33.272108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab6127-e2df-486e-b991-1cd3d65632ea admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:33.277534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e112b-5251-498c-bb4c-814454e3eb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:33.278151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e112b-5251-498c-bb4c-814454e3eb4e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:33.278431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e112b-5251-498c-bb4c-814454e3eb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:33.278743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e112b-5251-498c-bb4c-814454e3eb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:33.279413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e112b-5251-498c-bb4c-814454e3eb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:33.281021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e112b-5251-498c-bb4c-814454e3eb4e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:33.293504 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 966/1928] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:33.299740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bff70-fd92-4b0c-9abb-803eb863c870 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:33.310379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bff70-fd92-4b0c-9abb-803eb863c870 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:33.316089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9be033-c5ed-476f-b264-77e482808ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:33.316728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9be033-c5ed-476f-b264-77e482808ee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:33.317011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9be033-c5ed-476f-b264-77e482808ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:33.317286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9be033-c5ed-476f-b264-77e482808ee7 admin admin] PATH_INFO: `/v3/projects/e1c0b7e7d2b84f2ea70949692229b3bf/users/8ef5733978be4212aafa8b662a0fa572/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:33.332753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9be033-c5ed-476f-b264-77e482808ee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c0b7e7d2b84f2ea70949692229b3bf, user_id=8ef5733978be4212aafa8b662a0fa572, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:33.335774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9be033-c5ed-476f-b264-77e482808ee7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:33.351129 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 963/1929] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:33 2022] PUT /identity/v3/projects/e1c0b7e7d2b84f2ea70949692229b3bf/users/8ef5733978be4212aafa8b662a0fa572/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:33.361854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299834-1dad-4182-993e-cf0f4effbd43 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:33.362143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299834-1dad-4182-993e-cf0f4effbd43 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:33.362616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c299834-1dad-4182-993e-cf0f4effbd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:33.390211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6c299834-1dad-4182-993e-cf0f4effbd43 None None] MFA Rules not processed for user `8ef5733978be4212aafa8b662a0fa572`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:33.391481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6c299834-1dad-4182-993e-cf0f4effbd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:33.429075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:33.468055 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 967/1930] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:33.479623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9be033-c5ed-476f-b264-77e482808ee7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:33.481811 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-db9be033-c5ed-476f-b264-77e482808ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:33.497051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:33.551996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9be033-c5ed-476f-b264-77e482808ee7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:33.555637 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c2923-c48e-4e48-b8ce-edd9684aeb8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:33.556257 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c2923-c48e-4e48-b8ce-edd9684aeb8f service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:33.556557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c2923-c48e-4e48-b8ce-edd9684aeb8f service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:33.556828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c2923-c48e-4e48-b8ce-edd9684aeb8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:33.565498 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c2923-c48e-4e48-b8ce-edd9684aeb8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:33.566956 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c2923-c48e-4e48-b8ce-edd9684aeb8f service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:33.576355 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 964/1931] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:34.595501 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299834-1dad-4182-993e-cf0f4effbd43 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:34.598286 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6c299834-1dad-4182-993e-cf0f4effbd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:34.691740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c299834-1dad-4182-993e-cf0f4effbd43 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:34.697342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:34.700324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:34.702332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:34.702666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] PATH_INFO: `/v3/users/bee1c05fbfb94433a97928fa2eb5cd15` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:34.703921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bee1c05fbfb94433a97928fa2eb5cd15)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:34.705895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:34.765359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bee1c05fbfb94433a97928fa2eb5cd15'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:34.773864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bee1c05fbfb94433a97928fa2eb5cd15'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:34.774763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bee1c05fbfb94433a97928fa2eb5cd15'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:34.784735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bee1c05fbfb94433a97928fa2eb5cd15'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:34.799353 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f374a6a9-c53d-49f8-9041-bd164ca96674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bee1c05fbfb94433a97928fa2eb5cd15'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:34.806755 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 968/1932] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:34 2022] DELETE /identity/v3/users/bee1c05fbfb94433a97928fa2eb5cd15 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:34.982646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c2923-c48e-4e48-b8ce-edd9684aeb8f service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:34.995170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c2923-c48e-4e48-b8ce-edd9684aeb8f service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:35.095043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:35.095043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.095382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.095478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] PATH_INFO: `/v3/projects/c5f2240b2c6a43b8a810d84065ec9d44` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.097785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f2240b2c6a43b8a810d84065ec9d44)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:35.098698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:35.151032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f2240b2c6a43b8a810d84065ec9d44', 'request_id': 'req-e074c382-3e0c-406b-9995-e4af15e6e562'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:35.156738 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f2240b2c6a43b8a810d84065ec9d44', 'request_id': 'req-e074c382-3e0c-406b-9995-e4af15e6e562'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:35.163168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f2240b2c6a43b8a810d84065ec9d44', 'request_id': 'req-e074c382-3e0c-406b-9995-e4af15e6e562'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:35.178130 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] The token cache is being invalidate because project c5f2240b2c6a43b8a810d84065ec9d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:35.178392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:35.179033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e074c382-3e0c-406b-9995-e4af15e6e562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:35.180747 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 965/1933] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:34 2022] DELETE /identity/v3/projects/c5f2240b2c6a43b8a810d84065ec9d44 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:35.185582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0642f19-e984-44ad-a6ca-f6ae00ffcbbf None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.185922 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0642f19-e984-44ad-a6ca-f6ae00ffcbbf None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.186194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0642f19-e984-44ad-a6ca-f6ae00ffcbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.206574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d0642f19-e984-44ad-a6ca-f6ae00ffcbbf None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:35.207807 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d0642f19-e984-44ad-a6ca-f6ae00ffcbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:35.256552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:35.274924 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 969/1934] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:35.282925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a73171-5bc2-48cb-b21e-e647ffca7c93 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.283289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a73171-5bc2-48cb-b21e-e647ffca7c93 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.283656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a73171-5bc2-48cb-b21e-e647ffca7c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.305546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-68a73171-5bc2-48cb-b21e-e647ffca7c93 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:35.307180 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-68a73171-5bc2-48cb-b21e-e647ffca7c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:35.315933 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 966/1935] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:35.321119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0642f19-e984-44ad-a6ca-f6ae00ffcbbf None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:35.341807 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0642f19-e984-44ad-a6ca-f6ae00ffcbbf None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:35.347338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e9d3-8e9e-414f-8d14-edcdfb3a08db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:35.348074 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e9d3-8e9e-414f-8d14-edcdfb3a08db admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.348464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e9d3-8e9e-414f-8d14-edcdfb3a08db admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.348769 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e9d3-8e9e-414f-8d14-edcdfb3a08db admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.349502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67e9d3-8e9e-414f-8d14-edcdfb3a08db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:50:35.349982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67e9d3-8e9e-414f-8d14-edcdfb3a08db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:35.351813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67e9d3-8e9e-414f-8d14-edcdfb3a08db admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:35.374638 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 970/1936] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:50:35.379271 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a73171-5bc2-48cb-b21e-e647ffca7c93 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:35.388778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a73171-5bc2-48cb-b21e-e647ffca7c93 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:35.394917 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514b1ba-44ae-4ded-b33c-bdea33c1ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:35.395528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514b1ba-44ae-4ded-b33c-bdea33c1ad51 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.395870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514b1ba-44ae-4ded-b33c-bdea33c1ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.396062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514b1ba-44ae-4ded-b33c-bdea33c1ad51 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.396694 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514b1ba-44ae-4ded-b33c-bdea33c1ad51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:35.398604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514b1ba-44ae-4ded-b33c-bdea33c1ad51 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:35.433488 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 967/1937] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:50:35 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:35.434737 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e9d3-8e9e-414f-8d14-edcdfb3a08db admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:35.445382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e9d3-8e9e-414f-8d14-edcdfb3a08db admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:35.506117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22c2dd-129a-48de-ac90-9403fc388e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:35.506756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22c2dd-129a-48de-ac90-9403fc388e31 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.507043 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22c2dd-129a-48de-ac90-9403fc388e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.507311 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22c2dd-129a-48de-ac90-9403fc388e31 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.507940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22c2dd-129a-48de-ac90-9403fc388e31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:35.509880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22c2dd-129a-48de-ac90-9403fc388e31 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:35.533224 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 971/1938] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:50:35 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:50:35.537376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514b1ba-44ae-4ded-b33c-bdea33c1ad51 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:35.548038 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514b1ba-44ae-4ded-b33c-bdea33c1ad51 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:35.552555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6dc9f-07fb-406f-bef8-9d9235cf93f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:35.553148 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6dc9f-07fb-406f-bef8-9d9235cf93f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.553427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6dc9f-07fb-406f-bef8-9d9235cf93f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.553700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6dc9f-07fb-406f-bef8-9d9235cf93f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.554341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6dc9f-07fb-406f-bef8-9d9235cf93f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:35.555972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6dc9f-07fb-406f-bef8-9d9235cf93f8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:35.565804 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 968/1939] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:35.569731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22c2dd-129a-48de-ac90-9403fc388e31 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:35.580290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22c2dd-129a-48de-ac90-9403fc388e31 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:35.585464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d98159-ee99-4917-8d23-7788b20f047b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:35.586010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d98159-ee99-4917-8d23-7788b20f047b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.586301 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d98159-ee99-4917-8d23-7788b20f047b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.586562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d98159-ee99-4917-8d23-7788b20f047b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.587227 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d98159-ee99-4917-8d23-7788b20f047b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:35.588819 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d98159-ee99-4917-8d23-7788b20f047b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:35.599876 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 972/1940] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:50:35.604605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6dc9f-07fb-406f-bef8-9d9235cf93f8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:35.611638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:35.616154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6dc9f-07fb-406f-bef8-9d9235cf93f8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:35.622411 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985710bc-d65d-4f43-81a2-11ae1717120c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:35.623119 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985710bc-d65d-4f43-81a2-11ae1717120c admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.623400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985710bc-d65d-4f43-81a2-11ae1717120c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.623675 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985710bc-d65d-4f43-81a2-11ae1717120c admin admin] PATH_INFO: `/v3/projects/7fe93bf8d41a4b80b29a0cad6825f5b7/users/67292dd67aeb4cd3bb5c2e003871dd5c/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.639290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985710bc-d65d-4f43-81a2-11ae1717120c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fe93bf8d41a4b80b29a0cad6825f5b7, user_id=67292dd67aeb4cd3bb5c2e003871dd5c, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:35.641973 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985710bc-d65d-4f43-81a2-11ae1717120c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:35.656848 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 969/1941] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:50:35 2022] PUT /identity/v3/projects/7fe93bf8d41a4b80b29a0cad6825f5b7/users/67292dd67aeb4cd3bb5c2e003871dd5c/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:35.668919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe9147-b442-45a9-8a48-80f3f026bbbc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.669217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe9147-b442-45a9-8a48-80f3f026bbbc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.669481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe9147-b442-45a9-8a48-80f3f026bbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.698183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-acfe9147-b442-45a9-8a48-80f3f026bbbc None None] MFA Rules not processed for user `67292dd67aeb4cd3bb5c2e003871dd5c`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:35.699400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-acfe9147-b442-45a9-8a48-80f3f026bbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:35.781755 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 973/1942] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:35.794252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985710bc-d65d-4f43-81a2-11ae1717120c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:35.796432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-985710bc-d65d-4f43-81a2-11ae1717120c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:35.857996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985710bc-d65d-4f43-81a2-11ae1717120c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:35.861764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bb5ea-1ec3-4ce4-8e90-06d47c03f776 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b1c2ea68e4741728469ab7cc542b6d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:35.862379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bb5ea-1ec3-4ce4-8e90-06d47c03f776 service glance] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:35.862660 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bb5ea-1ec3-4ce4-8e90-06d47c03f776 service glance] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:35.862982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bb5ea-1ec3-4ce4-8e90-06d47c03f776 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:35.871658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bb5ea-1ec3-4ce4-8e90-06d47c03f776 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:35.873101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bb5ea-1ec3-4ce4-8e90-06d47c03f776 service glance] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:35.882652 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 970/1943] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:38.395990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe9147-b442-45a9-8a48-80f3f026bbbc None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:38.403171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-acfe9147-b442-45a9-8a48-80f3f026bbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:38.550779 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:38.611721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe9147-b442-45a9-8a48-80f3f026bbbc None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:38.611721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:38.612796 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:38.613415 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:38.613415 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:38.615282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:38.632880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:38.634502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:38.668827 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 974/1944] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:40.308691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f08b9-e311-4eb4-a9b8-8eddcab36f61 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:40.309292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f08b9-e311-4eb4-a9b8-8eddcab36f61 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:40.309292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f08b9-e311-4eb4-a9b8-8eddcab36f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:40.330747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9e3f08b9-e311-4eb4-a9b8-8eddcab36f61 None None] MFA Rules not processed for user `712e5cf4cc004918817ab1802b9c69d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:40.330747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9e3f08b9-e311-4eb4-a9b8-8eddcab36f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:40.399789 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 971/1945] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:44.546631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:44.548466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:44.626209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a452978-71d6-42a7-83a5-006b929a0c30 service placement] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:44.631003 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:44.631652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:44.631978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:44.632250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] PATH_INFO: `/v3/users/8ef5733978be4212aafa8b662a0fa572` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:44.633386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef5733978be4212aafa8b662a0fa572)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:44.635387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:44.693889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef5733978be4212aafa8b662a0fa572'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:44.708729 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef5733978be4212aafa8b662a0fa572'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:44.709578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef5733978be4212aafa8b662a0fa572'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:44.722639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef5733978be4212aafa8b662a0fa572'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:44.737390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef5733978be4212aafa8b662a0fa572'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:44.743904 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 975/1946] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:50:44 2022] DELETE /identity/v3/users/8ef5733978be4212aafa8b662a0fa572 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:44.748846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f29d-7c07-42d4-9e42-da2425822578 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:44.750585 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f29d-7c07-42d4-9e42-da2425822578 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:44.750973 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f29d-7c07-42d4-9e42-da2425822578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:44.771970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ecb6f29d-7c07-42d4-9e42-da2425822578 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:50:44.773644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ecb6f29d-7c07-42d4-9e42-da2425822578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:44.868028 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 972/1947] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:50:44.875885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:44.877699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:44.972844 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098350e-058f-48e2-906d-9589d05d2265 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:44.991319 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddd6e3-8b48-4d62-b9fd-507ed25bd125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:44.991963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddd6e3-8b48-4d62-b9fd-507ed25bd125 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:44.992233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddd6e3-8b48-4d62-b9fd-507ed25bd125 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:44.992836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddd6e3-8b48-4d62-b9fd-507ed25bd125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:45.014829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddd6e3-8b48-4d62-b9fd-507ed25bd125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:45.016791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fddd6e3-8b48-4d62-b9fd-507ed25bd125 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:45.039007 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 976/1948] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:50:45.780622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6f29d-7c07-42d4-9e42-da2425822578 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:50:45.782226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ecb6f29d-7c07-42d4-9e42-da2425822578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:50:45.795641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6f29d-7c07-42d4-9e42-da2425822578 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:50:45.801161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:50:45.801837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:50:45.802168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:50:45.802450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] PATH_INFO: `/v3/projects/e1c0b7e7d2b84f2ea70949692229b3bf` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:50:45.803703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c0b7e7d2b84f2ea70949692229b3bf)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:50:45.805604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:50:45.842880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:45.870621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c0b7e7d2b84f2ea70949692229b3bf', 'request_id': 'req-241407c8-ae8c-4793-880b-8f00ea318447'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:45.876483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c0b7e7d2b84f2ea70949692229b3bf', 'request_id': 'req-241407c8-ae8c-4793-880b-8f00ea318447'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:45.883515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c0b7e7d2b84f2ea70949692229b3bf', 'request_id': 'req-241407c8-ae8c-4793-880b-8f00ea318447'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:45.900931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] The token cache is being invalidate because project e1c0b7e7d2b84f2ea70949692229b3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:50:45.901290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:45.902069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-241407c8-ae8c-4793-880b-8f00ea318447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:50:45.903710 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 973/1949] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:50:45 2022] DELETE /identity/v3/projects/e1c0b7e7d2b84f2ea70949692229b3bf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:50:55.026202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:50:55.895728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:07.278015 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddd6e3-8b48-4d62-b9fd-507ed25bd125 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:51:07.279627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3fddd6e3-8b48-4d62-b9fd-507ed25bd125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:07.295374 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddd6e3-8b48-4d62-b9fd-507ed25bd125 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:51:07.355492 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:51:07.356104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:07.356386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:07.356687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] PATH_INFO: `/v3/users/67292dd67aeb4cd3bb5c2e003871dd5c` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:07.357976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] RBAC: Authorizing `identity:delete_user(user_id=67292dd67aeb4cd3bb5c2e003871dd5c)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:51:07.360005 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:51:07.408206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67292dd67aeb4cd3bb5c2e003871dd5c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:07.415223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67292dd67aeb4cd3bb5c2e003871dd5c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:07.415905 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67292dd67aeb4cd3bb5c2e003871dd5c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:07.423694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67292dd67aeb4cd3bb5c2e003871dd5c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:07.435972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67292dd67aeb4cd3bb5c2e003871dd5c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:07.441900 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 977/1950] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:51:07 2022] DELETE /identity/v3/users/67292dd67aeb4cd3bb5c2e003871dd5c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:51:07.446555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca83837-592d-4927-b7aa-09b5be16d375 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:07.447078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca83837-592d-4927-b7aa-09b5be16d375 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:07.447170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca83837-592d-4927-b7aa-09b5be16d375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:07.468267 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-fca83837-592d-4927-b7aa-09b5be16d375 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:51:07.469530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fca83837-592d-4927-b7aa-09b5be16d375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:07.533688 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 974/1951] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:51:07.543917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:51:07.545586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:07.604010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a713b-47d8-4ebf-84d4-8d580e27f51b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:51:07.607910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec23b6-69f4-4745-a87a-1c090b7ee24f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:51:07.608532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec23b6-69f4-4745-a87a-1c090b7ee24f service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:07.608813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec23b6-69f4-4745-a87a-1c090b7ee24f service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:07.609076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec23b6-69f4-4745-a87a-1c090b7ee24f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:07.619587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec23b6-69f4-4745-a87a-1c090b7ee24f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:51:07.621166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec23b6-69f4-4745-a87a-1c090b7ee24f service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:51:07.633190 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 978/1952] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:51:07.911851 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca83837-592d-4927-b7aa-09b5be16d375 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:51:07.913418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fca83837-592d-4927-b7aa-09b5be16d375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:07.932391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca83837-592d-4927-b7aa-09b5be16d375 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:51:07.943839 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:51:07.944493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:07.944790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:07.945056 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] PATH_INFO: `/v3/projects/7fe93bf8d41a4b80b29a0cad6825f5b7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:07.946253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fe93bf8d41a4b80b29a0cad6825f5b7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:51:07.948219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:51:07.996502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fe93bf8d41a4b80b29a0cad6825f5b7', 'request_id': 'req-76d08f1f-97ba-4147-aac5-9466befd5dff'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:08.001992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fe93bf8d41a4b80b29a0cad6825f5b7', 'request_id': 'req-76d08f1f-97ba-4147-aac5-9466befd5dff'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:08.008178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fe93bf8d41a4b80b29a0cad6825f5b7', 'request_id': 'req-76d08f1f-97ba-4147-aac5-9466befd5dff'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:08.022428 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] The token cache is being invalidate because project 7fe93bf8d41a4b80b29a0cad6825f5b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:51:08.022777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:08.023401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:51:08.024894 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 975/1953] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:51:07 2022] DELETE /identity/v3/projects/7fe93bf8d41a4b80b29a0cad6825f5b7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:51:09.986355 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1faa08-49a7-47ea-b337-49f44c2c7bdb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:09.986899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1faa08-49a7-47ea-b337-49f44c2c7bdb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:09.987054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1faa08-49a7-47ea-b337-49f44c2c7bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:10.007257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bb1faa08-49a7-47ea-b337-49f44c2c7bdb None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:51:10.008462 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bb1faa08-49a7-47ea-b337-49f44c2c7bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:10.026200 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:10.065447 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 979/1954] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:51:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:51:10.077651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:51:10.079142 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:10.151762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d08f1f-97ba-4147-aac5-9466befd5dff admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:51:10.157979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0172025-a30c-43a4-8318-c8921d75b458 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:51:10.158583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0172025-a30c-43a4-8318-c8921d75b458 service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:10.158951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0172025-a30c-43a4-8318-c8921d75b458 service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:10.159222 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0172025-a30c-43a4-8318-c8921d75b458 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:10.168531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0172025-a30c-43a4-8318-c8921d75b458 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:51:10.170123 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0172025-a30c-43a4-8318-c8921d75b458 service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:51:10.178849 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 976/1955] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:51:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 08:51:20.058575 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=15 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:20.175377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:25.118133 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe050d6-2828-4c90-bbf0-eb9395c9e1e0 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:25.118775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe050d6-2828-4c90-bbf0-eb9395c9e1e0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:25.118775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe050d6-2828-4c90-bbf0-eb9395c9e1e0 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:25.120464 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 980/1956] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:51:25.126622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81c7ea-541d-4f89-bf0a-60d82fb1cbe0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:25.127116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81c7ea-541d-4f89-bf0a-60d82fb1cbe0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:25.127432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81c7ea-541d-4f89-bf0a-60d82fb1cbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:25.148698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7d81c7ea-541d-4f89-bf0a-60d82fb1cbe0 None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:51:25.149955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7d81c7ea-541d-4f89-bf0a-60d82fb1cbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:25.217640 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 977/1957] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:51:25.229095 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe050d6-2828-4c90-bbf0-eb9395c9e1e0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:51:25.230942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dbe050d6-2828-4c90-bbf0-eb9395c9e1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:25.287596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe050d6-2828-4c90-bbf0-eb9395c9e1e0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:51:25.291256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9ae47-e9a7-43c5-892b-5f5f365d85e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b1c2ea68e4741728469ab7cc542b6d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:51:25.291835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9ae47-e9a7-43c5-892b-5f5f365d85e7 service glance] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:25.292110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9ae47-e9a7-43c5-892b-5f5f365d85e7 service glance] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:25.292378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9ae47-e9a7-43c5-892b-5f5f365d85e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:25.302489 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9ae47-e9a7-43c5-892b-5f5f365d85e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:51:25.303971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9ae47-e9a7-43c5-892b-5f5f365d85e7 service glance] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:51:25.315033 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 981/1958] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:51:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:51:25.399141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe239e56-50f0-4b72-b65c-aad757c0c1fc None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:25.399542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe239e56-50f0-4b72-b65c-aad757c0c1fc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:25.399982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe239e56-50f0-4b72-b65c-aad757c0c1fc None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:25.402120 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 978/1959] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:51:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:51:25.409155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501c5c3-4bbb-4204-bbba-7187aa276eea None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:25.409438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501c5c3-4bbb-4204-bbba-7187aa276eea None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:25.409713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501c5c3-4bbb-4204-bbba-7187aa276eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:25.429430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4501c5c3-4bbb-4204-bbba-7187aa276eea None None] MFA Rules not processed for user `2254623f523c48f0aa454b85be41df95`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:51:25.430626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4501c5c3-4bbb-4204-bbba-7187aa276eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:25.482891 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 982/1960] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:51:25.494867 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe239e56-50f0-4b72-b65c-aad757c0c1fc None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:51:25.496620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fe239e56-50f0-4b72-b65c-aad757c0c1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:25.510071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe239e56-50f0-4b72-b65c-aad757c0c1fc None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:51:25.513668 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddeb30-807f-4a28-947c-6e83976c1071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:51:25.514259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddeb30-807f-4a28-947c-6e83976c1071 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:25.514535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddeb30-807f-4a28-947c-6e83976c1071 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:25.514834 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddeb30-807f-4a28-947c-6e83976c1071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:25.521797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddeb30-807f-4a28-947c-6e83976c1071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:51:25.523240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddeb30-807f-4a28-947c-6e83976c1071 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:51:25.531087 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 979/1961] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:51:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:51:25.628878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e3f41-bea0-41b9-96c7-b01a08d662b2 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:25.629152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e3f41-bea0-41b9-96c7-b01a08d662b2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:25.629435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e3f41-bea0-41b9-96c7-b01a08d662b2 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:25.631063 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 983/1962] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:51:25.636726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc4e05-2117-4865-941f-269ba3b08c2a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:25.637031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc4e05-2117-4865-941f-269ba3b08c2a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:25.637271 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc4e05-2117-4865-941f-269ba3b08c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:25.658245 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a6fc4e05-2117-4865-941f-269ba3b08c2a None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:51:25.659475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a6fc4e05-2117-4865-941f-269ba3b08c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:25.668928 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 980/1963] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:51:25.680016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e3f41-bea0-41b9-96c7-b01a08d662b2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:51:25.691998 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e3f41-bea0-41b9-96c7-b01a08d662b2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:51:25.696553 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f6601-5646-426d-b4cf-1c54cf34efea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:51:25.697154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f6601-5646-426d-b4cf-1c54cf34efea service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:25.697436 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f6601-5646-426d-b4cf-1c54cf34efea service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:25.697730 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f6601-5646-426d-b4cf-1c54cf34efea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:25.708304 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f6601-5646-426d-b4cf-1c54cf34efea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:51:25.709738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f6601-5646-426d-b4cf-1c54cf34efea service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:51:25.720494 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 984/1964] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:51:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:51:28.360738 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41249f22-cd5c-4fa2-b48e-05c6e3dcc723 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:28.361259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41249f22-cd5c-4fa2-b48e-05c6e3dcc723 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:28.361259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41249f22-cd5c-4fa2-b48e-05c6e3dcc723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:28.381582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-41249f22-cd5c-4fa2-b48e-05c6e3dcc723 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:51:28.382890 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-41249f22-cd5c-4fa2-b48e-05c6e3dcc723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:28.391705 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 981/1965] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:51:28.404112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f6601-5646-426d-b4cf-1c54cf34efea service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:51:28.417434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f6601-5646-426d-b4cf-1c54cf34efea service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:51:28.424228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f92c36-32a9-4f21-b4a0-ad1157ca660f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:51:28.424819 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f92c36-32a9-4f21-b4a0-ad1157ca660f service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:28.425094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f92c36-32a9-4f21-b4a0-ad1157ca660f service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:28.425360 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f92c36-32a9-4f21-b4a0-ad1157ca660f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:28.434039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f92c36-32a9-4f21-b4a0-ad1157ca660f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:51:28.435627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f92c36-32a9-4f21-b4a0-ad1157ca660f service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:51:28.444045 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 985/1966] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:51:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:51:38.375904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=43 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:38.440476 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=61 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:51:56.658703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b4d29-a0b0-4dd9-8880-bbfaf47929fa None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:56.659256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b4d29-a0b0-4dd9-8880-bbfaf47929fa None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:56.659354 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b4d29-a0b0-4dd9-8880-bbfaf47929fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:56.679601 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b56b4d29-a0b0-4dd9-8880-bbfaf47929fa None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:51:56.680808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b56b4d29-a0b0-4dd9-8880-bbfaf47929fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:51:56.689196 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 982/1967] 10.222.0.26 () {58 vars in 967 bytes} [Mon Oct 24 08:51:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:51:56.701798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f92c36-32a9-4f21-b4a0-ad1157ca660f service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:51:56.712979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f92c36-32a9-4f21-b4a0-ad1157ca660f service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:51:56.718498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30189725-ec16-42be-9085-c1382094a486 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:51:56.719137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30189725-ec16-42be-9085-c1382094a486 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:51:56.719417 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30189725-ec16-42be-9085-c1382094a486 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:51:56.719694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30189725-ec16-42be-9085-c1382094a486 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:51:56.727721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30189725-ec16-42be-9085-c1382094a486 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:51:56.729303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30189725-ec16-42be-9085-c1382094a486 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:51:56.737260 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 986/1968] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:51:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:52:06.673766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:06.734051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=6 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:52:36.653878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b4d29-a0b0-4dd9-8880-bbfaf47929fa None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:52:36.655370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b56b4d29-a0b0-4dd9-8880-bbfaf47929fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:52:36.667005 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b4d29-a0b0-4dd9-8880-bbfaf47929fa None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:52:36.671097 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6dc40-ea18-47d8-9077-29d50ca719e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:52:36.671750 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6dc40-ea18-47d8-9077-29d50ca719e0 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:52:36.672024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6dc40-ea18-47d8-9077-29d50ca719e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:52:36.672291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6dc40-ea18-47d8-9077-29d50ca719e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:52:36.674018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0ff6dc40-ea18-47d8-9077-29d50ca719e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:52:36.688071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6dc40-ea18-47d8-9077-29d50ca719e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:52:36.689686 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6dc40-ea18-47d8-9077-29d50ca719e0 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:52:36.700987 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 983/1969] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:52:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:52:38.131716 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30189725-ec16-42be-9085-c1382094a486 service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:52:38.143568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30189725-ec16-42be-9085-c1382094a486 service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:52:38.148652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b32993-55b7-4575-9f77-10eb55c6ab94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:52:38.149264 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b32993-55b7-4575-9f77-10eb55c6ab94 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:52:38.149541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b32993-55b7-4575-9f77-10eb55c6ab94 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:52:38.149811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b32993-55b7-4575-9f77-10eb55c6ab94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:52:38.151948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-02b32993-55b7-4575-9f77-10eb55c6ab94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:52:38.168965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b32993-55b7-4575-9f77-10eb55c6ab94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:52:38.170479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b32993-55b7-4575-9f77-10eb55c6ab94 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:52:38.182939 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 987/1970] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:52:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:52:42.185628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af937a25-8105-47a8-b176-2528887ab87f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:52:42.186093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af937a25-8105-47a8-b176-2528887ab87f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:52:42.186093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af937a25-8105-47a8-b176-2528887ab87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:52:42.207236 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-af937a25-8105-47a8-b176-2528887ab87f None None] MFA Rules not processed for user `8695364e58b04d829b12ae1b49757536`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:52:42.208420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-af937a25-8105-47a8-b176-2528887ab87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:52:42.267100 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 984/1971] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:52:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:52:44.994421 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e613e-17d3-4735-97cb-75394091ee88 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:52:44.994987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e613e-17d3-4735-97cb-75394091ee88 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:52:44.994987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e613e-17d3-4735-97cb-75394091ee88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:52:45.014508 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fd7e613e-17d3-4735-97cb-75394091ee88 None None] MFA Rules not processed for user `712e5cf4cc004918817ab1802b9c69d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:52:45.015786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fd7e613e-17d3-4735-97cb-75394091ee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:52:45.071222 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 988/1972] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:52:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:52:51.626944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af937a25-8105-47a8-b176-2528887ab87f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:52:51.629935 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-af937a25-8105-47a8-b176-2528887ab87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:52:51.655529 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af937a25-8105-47a8-b176-2528887ab87f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:52:51.662358 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabde80-8d2e-4940-8940-0c2843bf8e57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:52:51.663020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabde80-8d2e-4940-8940-0c2843bf8e57 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:52:51.663282 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabde80-8d2e-4940-8940-0c2843bf8e57 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:52:51.663556 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabde80-8d2e-4940-8940-0c2843bf8e57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:52:51.665323 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2dabde80-8d2e-4940-8940-0c2843bf8e57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:52:51.676249 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabde80-8d2e-4940-8940-0c2843bf8e57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:52:51.677858 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabde80-8d2e-4940-8940-0c2843bf8e57 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:52:51.687114 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 985/1973] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:52:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:52:55.064536 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=31 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:00.755565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e613e-17d3-4735-97cb-75394091ee88 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:00.769682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e613e-17d3-4735-97cb-75394091ee88 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:00.777015 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e48fd-c21f-4b50-af6b-b519aa319391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:00.777772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e48fd-c21f-4b50-af6b-b519aa319391 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:00.777895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e48fd-c21f-4b50-af6b-b519aa319391 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:00.778180 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e48fd-c21f-4b50-af6b-b519aa319391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:00.780070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-302e48fd-c21f-4b50-af6b-b519aa319391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:00.839300 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e48fd-c21f-4b50-af6b-b519aa319391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:00.840856 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e48fd-c21f-4b50-af6b-b519aa319391 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:00.850170 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 989/1974] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:01.683010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=37 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:01.997419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabde80-8d2e-4940-8940-0c2843bf8e57 service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:02.008223 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabde80-8d2e-4940-8940-0c2843bf8e57 service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:02.013238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973016af-0e9d-416a-b22f-55d464148d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:02.014102 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973016af-0e9d-416a-b22f-55d464148d31 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:02.014371 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973016af-0e9d-416a-b22f-55d464148d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:02.014970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973016af-0e9d-416a-b22f-55d464148d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:02.016924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-973016af-0e9d-416a-b22f-55d464148d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:02.027314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973016af-0e9d-416a-b22f-55d464148d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:02.028887 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973016af-0e9d-416a-b22f-55d464148d31 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:02.037392 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 986/1975] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:10.247021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba55c6d-3a73-43a9-93a3-f1221405228b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:10.247558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba55c6d-3a73-43a9-93a3-f1221405228b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:10.247558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba55c6d-3a73-43a9-93a3-f1221405228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:10.268439 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-aba55c6d-3a73-43a9-93a3-f1221405228b None None] MFA Rules not processed for user `43af1b9b66804624aff6eef2f435d581`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:53:10.269639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aba55c6d-3a73-43a9-93a3-f1221405228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:10.277482 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 990/1976] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:53:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:53:10.837406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973016af-0e9d-416a-b22f-55d464148d31 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:10.846802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973016af-0e9d-416a-b22f-55d464148d31 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:10.851882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:10.852478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:10.852723 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:10.853021 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:10.855264 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:10.870026 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:10.872119 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:10.882131 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 987/1977] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:11.212131 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba55c6d-3a73-43a9-93a3-f1221405228b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:11.213692 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aba55c6d-3a73-43a9-93a3-f1221405228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:11.291804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba55c6d-3a73-43a9-93a3-f1221405228b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:11.297868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e0dae-4308-4435-ae2c-f5b73fd7a27e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:11.298621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e0dae-4308-4435-ae2c-f5b73fd7a27e service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:11.298938 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e0dae-4308-4435-ae2c-f5b73fd7a27e service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:11.299201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e0dae-4308-4435-ae2c-f5b73fd7a27e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:11.301187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-407e0dae-4308-4435-ae2c-f5b73fd7a27e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:11.313115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e0dae-4308-4435-ae2c-f5b73fd7a27e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:11.314750 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e0dae-4308-4435-ae2c-f5b73fd7a27e service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:11.325786 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 991/1978] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:14.239819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:14.242109 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:14.260086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c03ba2-b830-43f1-a8d8-5bc8bd4894f8 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:14.267755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b76e2c-95b0-4212-a5b8-d6fe033504c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:14.268639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b76e2c-95b0-4212-a5b8-d6fe033504c6 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:14.269069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b76e2c-95b0-4212-a5b8-d6fe033504c6 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:14.269512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b76e2c-95b0-4212-a5b8-d6fe033504c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:14.271829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-79b76e2c-95b0-4212-a5b8-d6fe033504c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:14.286360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b76e2c-95b0-4212-a5b8-d6fe033504c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:14.287932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b76e2c-95b0-4212-a5b8-d6fe033504c6 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:14.306542 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 988/1979] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:20.980149 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e0dae-4308-4435-ae2c-f5b73fd7a27e service placement] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:20.993217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e0dae-4308-4435-ae2c-f5b73fd7a27e service placement] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:20.999484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7c6fc-0a03-4ad7-9298-1975c581e8b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:21.000121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7c6fc-0a03-4ad7-9298-1975c581e8b1 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:21.000412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7c6fc-0a03-4ad7-9298-1975c581e8b1 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:21.000683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7c6fc-0a03-4ad7-9298-1975c581e8b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:21.002508 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-90c7c6fc-0a03-4ad7-9298-1975c581e8b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:21.016393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7c6fc-0a03-4ad7-9298-1975c581e8b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:21.018018 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7c6fc-0a03-4ad7-9298-1975c581e8b1 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:21.029132 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 992/1980] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:22.548597 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3c767-8ea8-4092-abbb-7ec7f0390b0d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:22.549061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3c767-8ea8-4092-abbb-7ec7f0390b0d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:22.549165 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3c767-8ea8-4092-abbb-7ec7f0390b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:22.569392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-69c3c767-8ea8-4092-abbb-7ec7f0390b0d None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:53:22.570585 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-69c3c767-8ea8-4092-abbb-7ec7f0390b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:22.579014 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 989/1981] 10.222.0.26 () {58 vars in 967 bytes} [Mon Oct 24 08:53:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:53:22.626537 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7c6fc-0a03-4ad7-9298-1975c581e8b1 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:22.638177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7c6fc-0a03-4ad7-9298-1975c581e8b1 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:22.644521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf6555-fd93-4a2e-a3a1-993b6c0eb77f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:22.645126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf6555-fd93-4a2e-a3a1-993b6c0eb77f service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:22.645397 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf6555-fd93-4a2e-a3a1-993b6c0eb77f service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:22.645648 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf6555-fd93-4a2e-a3a1-993b6c0eb77f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:22.657205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf6555-fd93-4a2e-a3a1-993b6c0eb77f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:22.658856 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf6555-fd93-4a2e-a3a1-993b6c0eb77f service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:22.667288 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 993/1982] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:53:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:53:26.311996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3c767-8ea8-4092-abbb-7ec7f0390b0d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:26.313415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-69c3c767-8ea8-4092-abbb-7ec7f0390b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:26.331115 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3c767-8ea8-4092-abbb-7ec7f0390b0d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:26.337169 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:26.338019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:26.338481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:26.338962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:26.340750 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:26.354837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:26.357116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:26.370069 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 990/1983] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:28.127781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf6555-fd93-4a2e-a3a1-993b6c0eb77f service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:28.129522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4faf6555-fd93-4a2e-a3a1-993b6c0eb77f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:28.136400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=76 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:28.196972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf6555-fd93-4a2e-a3a1-993b6c0eb77f service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:28.200916 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a1537-de92-4362-aca5-02a4cb488404 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:28.201538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a1537-de92-4362-aca5-02a4cb488404 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:28.201971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a1537-de92-4362-aca5-02a4cb488404 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:28.202386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a1537-de92-4362-aca5-02a4cb488404 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:28.205332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cc1a1537-de92-4362-aca5-02a4cb488404 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:28.293121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a1537-de92-4362-aca5-02a4cb488404 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:28.294491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a1537-de92-4362-aca5-02a4cb488404 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:28.321572 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 994/1984] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:29.640647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:29.643158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:29.712218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e51ae-1cd7-42e9-bf0a-d5a60e000500 service placement] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:29.716687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:29.717330 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:29.717608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:29.717883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] PATH_INFO: `/v3/users/1aa023d6ce6b45efa086c40c292f4ad0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:29.722642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa023d6ce6b45efa086c40c292f4ad0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:29.722642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:29.769316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa023d6ce6b45efa086c40c292f4ad0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:29.770002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa023d6ce6b45efa086c40c292f4ad0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:29.777230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa023d6ce6b45efa086c40c292f4ad0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:29.785944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa023d6ce6b45efa086c40c292f4ad0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:29.798702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa023d6ce6b45efa086c40c292f4ad0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:29.805397 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 991/1985] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:53:29 2022] DELETE /identity/v3/users/1aa023d6ce6b45efa086c40c292f4ad0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:53:30.053163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a1537-de92-4362-aca5-02a4cb488404 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:30.064554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a1537-de92-4362-aca5-02a4cb488404 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:30.123846 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:30.124492 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:30.124761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:30.125016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] PATH_INFO: `/v3/projects/238b1450574c411682599a39ad0a8371` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:30.126223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=238b1450574c411682599a39ad0a8371)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:30.128233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:30.175446 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '238b1450574c411682599a39ad0a8371', 'request_id': 'req-043dc851-6017-4ea4-b369-b41adad4bb08'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.180914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '238b1450574c411682599a39ad0a8371', 'request_id': 'req-043dc851-6017-4ea4-b369-b41adad4bb08'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.181998 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:30.188630 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '238b1450574c411682599a39ad0a8371', 'request_id': 'req-043dc851-6017-4ea4-b369-b41adad4bb08'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.202843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] The token cache is being invalidate because project 238b1450574c411682599a39ad0a8371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:53:30.203160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.203805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.205426 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 995/1986] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:53:30 2022] DELETE /identity/v3/projects/238b1450574c411682599a39ad0a8371 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:53:30.209654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:30.211092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:30.224257 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b40f-8559-47a2-ac33-cba039a34b63 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:30.241082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:30.282360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:30.283068 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:30.283373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:30.283641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] PATH_INFO: `/v3/users/e89c572bdd364c0b9c9dc1b843daffdf` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:30.285350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e89c572bdd364c0b9c9dc1b843daffdf)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:30.286741 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:30.336714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e89c572bdd364c0b9c9dc1b843daffdf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.337376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e89c572bdd364c0b9c9dc1b843daffdf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.344618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e89c572bdd364c0b9c9dc1b843daffdf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.353214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e89c572bdd364c0b9c9dc1b843daffdf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.367199 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6a69f7ac-9f37-4aa1-a111-633bd7c4401f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e89c572bdd364c0b9c9dc1b843daffdf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.373249 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 992/1987] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:53:30 2022] DELETE /identity/v3/users/e89c572bdd364c0b9c9dc1b843daffdf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:53:30.585601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:30.587842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:30.604075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dc851-6017-4ea4-b369-b41adad4bb08 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:30.664216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:30.664865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:30.665143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:30.665417 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] PATH_INFO: `/v3/projects/0043aeec1ea646f3be919354aaa96452` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:30.666537 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0043aeec1ea646f3be919354aaa96452)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:30.668485 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:30.718027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0043aeec1ea646f3be919354aaa96452', 'request_id': 'req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.724514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0043aeec1ea646f3be919354aaa96452', 'request_id': 'req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.731328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0043aeec1ea646f3be919354aaa96452', 'request_id': 'req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.746025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] The token cache is being invalidate because project 0043aeec1ea646f3be919354aaa96452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:53:30.746231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.746892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:53:30.748443 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 996/1988] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:53:30 2022] DELETE /identity/v3/projects/0043aeec1ea646f3be919354aaa96452 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:53:34.282621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae160c-a6bc-4dbf-acb9-ff5046f6c41b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:34.283221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae160c-a6bc-4dbf-acb9-ff5046f6c41b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:34.283301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae160c-a6bc-4dbf-acb9-ff5046f6c41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:34.303444 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3eae160c-a6bc-4dbf-acb9-ff5046f6c41b None None] MFA Rules not processed for user `7d793a1f26a3467ea8119e27e3394371`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:53:34.304664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3eae160c-a6bc-4dbf-acb9-ff5046f6c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:34.359890 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 993/1989] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:53:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:53:34.671114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:34.672695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:34.735663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8086013-fc1b-42d5-b4d0-408f6fe6bb90 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:34.739980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:34.740587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:34.740862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:34.741136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:34.742895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:34.752738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:34.754158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:34.762104 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 997/1990] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:43.875999 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae160c-a6bc-4dbf-acb9-ff5046f6c41b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:43.877447 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3eae160c-a6bc-4dbf-acb9-ff5046f6c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:43.930553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:43.960414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae160c-a6bc-4dbf-acb9-ff5046f6c41b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:43.966777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:43.967435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:43.967709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:43.967977 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:43.969830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:43.981788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:43.983385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:43.993882 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 994/1991] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:53:44.758493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:53.981158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:53:53.983805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:53.989279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=14 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:53:54.041484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf8fe2-c4b3-406e-ad2c-9f52fead6d5e service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:53:54.045031 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:53:54.045950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:53:54.046230 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:53:54.046503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:53:54.048411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:53:54.112087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:53:54.113531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:53:54.123149 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 998/1992] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:53:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:04.118748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=48 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:13.023216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:13.024916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:13.041903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55bad5-a882-40a9-b4d2-b034c324933f service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:13.047022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:13.047648 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:13.047916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:13.048217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:13.050555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:13.121571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:13.123039 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:13.133147 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 995/1993] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:16.494003 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:16.495493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:16.553145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:16.554657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:16.572098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b105c63-c5de-4864-880b-fae33603ec0e service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:16.572263 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c5519-e44b-4474-8d91-648eb6333862 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:16.577237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:16.577474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a19d0b-06b2-40a3-9306-15b48b07ebab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:16.577876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:16.578116 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a19d0b-06b2-40a3-9306-15b48b07ebab service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:16.578213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:16.578410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] PATH_INFO: `/v3/users/8695364e58b04d829b12ae1b49757536` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:16.578515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a19d0b-06b2-40a3-9306-15b48b07ebab service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:16.578687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a19d0b-06b2-40a3-9306-15b48b07ebab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:16.579657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8695364e58b04d829b12ae1b49757536)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:16.580590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-71a19d0b-06b2-40a3-9306-15b48b07ebab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:16.581667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:16.593977 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a19d0b-06b2-40a3-9306-15b48b07ebab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:16.596043 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a19d0b-06b2-40a3-9306-15b48b07ebab service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:16.607632 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 999/1994] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:16.631739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8695364e58b04d829b12ae1b49757536'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.632391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8695364e58b04d829b12ae1b49757536'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.640071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8695364e58b04d829b12ae1b49757536'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.647990 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8695364e58b04d829b12ae1b49757536'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.660672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f3a596e7-9341-402d-97dd-e99d1436280c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8695364e58b04d829b12ae1b49757536'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.667450 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 996/1995] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:54:16 2022] DELETE /identity/v3/users/8695364e58b04d829b12ae1b49757536 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:16.836578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a19d0b-06b2-40a3-9306-15b48b07ebab service placement] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:16.851768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a19d0b-06b2-40a3-9306-15b48b07ebab service placement] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:16.913761 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:16.914409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:16.914687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:16.914997 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] PATH_INFO: `/v3/projects/f063c719bcda47e6874437d32d284815` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:16.916167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f063c719bcda47e6874437d32d284815)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:16.918141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:16.966700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f063c719bcda47e6874437d32d284815', 'request_id': 'req-5b1790a3-b157-4824-a61f-f31dde554bd5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.972869 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f063c719bcda47e6874437d32d284815', 'request_id': 'req-5b1790a3-b157-4824-a61f-f31dde554bd5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.979183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f063c719bcda47e6874437d32d284815', 'request_id': 'req-5b1790a3-b157-4824-a61f-f31dde554bd5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.993307 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] The token cache is being invalidate because project f063c719bcda47e6874437d32d284815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:54:16.993602 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.994310 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5b1790a3-b157-4824-a61f-f31dde554bd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:16.995928 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1000/1996] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:54:16 2022] DELETE /identity/v3/projects/f063c719bcda47e6874437d32d284815 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:17.000942 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c9f9-bde3-41de-8479-b2a978840fef None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.001278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c9f9-bde3-41de-8479-b2a978840fef None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.001558 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c9f9-bde3-41de-8479-b2a978840fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.021676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3cb6c9f9-bde3-41de-8479-b2a978840fef None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:17.022908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3cb6c9f9-bde3-41de-8479-b2a978840fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:17.084518 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 997/1997] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:17.092651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cc3e6-0cb3-41a2-b027-f0e3cca6d7e9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.092961 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cc3e6-0cb3-41a2-b027-f0e3cca6d7e9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.093224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cc3e6-0cb3-41a2-b027-f0e3cca6d7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.116238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-022cc3e6-0cb3-41a2-b027-f0e3cca6d7e9 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:17.117481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-022cc3e6-0cb3-41a2-b027-f0e3cca6d7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:17.126838 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1001/1998] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:17.131355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c9f9-bde3-41de-8479-b2a978840fef None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.141551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c9f9-bde3-41de-8479-b2a978840fef None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.146747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f39703-1ff6-4e9f-a2ac-8b7e9142b641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.147374 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f39703-1ff6-4e9f-a2ac-8b7e9142b641 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.147653 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f39703-1ff6-4e9f-a2ac-8b7e9142b641 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.147931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f39703-1ff6-4e9f-a2ac-8b7e9142b641 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.148610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f39703-1ff6-4e9f-a2ac-8b7e9142b641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:54:17.148912 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f39703-1ff6-4e9f-a2ac-8b7e9142b641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.150510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f39703-1ff6-4e9f-a2ac-8b7e9142b641 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.152152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:17.163790 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 998/1999] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:54:17.167730 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cc3e6-0cb3-41a2-b027-f0e3cca6d7e9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.178039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cc3e6-0cb3-41a2-b027-f0e3cca6d7e9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.183930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704daaf2-5aa9-4bfd-ad7b-27d8be09aed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.184529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704daaf2-5aa9-4bfd-ad7b-27d8be09aed7 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.184804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704daaf2-5aa9-4bfd-ad7b-27d8be09aed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.185072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704daaf2-5aa9-4bfd-ad7b-27d8be09aed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.185707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704daaf2-5aa9-4bfd-ad7b-27d8be09aed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.187647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704daaf2-5aa9-4bfd-ad7b-27d8be09aed7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.199143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:17.217503 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1002/2000] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:54:17 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:54:17.221967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f39703-1ff6-4e9f-a2ac-8b7e9142b641 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.232638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f39703-1ff6-4e9f-a2ac-8b7e9142b641 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.293014 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b17621-47ba-4f4e-8d47-cac76ad061c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.293628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b17621-47ba-4f4e-8d47-cac76ad061c3 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.293909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b17621-47ba-4f4e-8d47-cac76ad061c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.294182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b17621-47ba-4f4e-8d47-cac76ad061c3 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.294871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b17621-47ba-4f4e-8d47-cac76ad061c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.297114 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b17621-47ba-4f4e-8d47-cac76ad061c3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.319296 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 999/2001] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:54:17 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:54:17.323513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704daaf2-5aa9-4bfd-ad7b-27d8be09aed7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.334375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704daaf2-5aa9-4bfd-ad7b-27d8be09aed7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.339753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896125ca-a21e-4305-910e-20a94325cc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.340374 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896125ca-a21e-4305-910e-20a94325cc9e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.340677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896125ca-a21e-4305-910e-20a94325cc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.340967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896125ca-a21e-4305-910e-20a94325cc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.341643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896125ca-a21e-4305-910e-20a94325cc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.343329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896125ca-a21e-4305-910e-20a94325cc9e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.354008 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1003/2002] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:17.357961 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b17621-47ba-4f4e-8d47-cac76ad061c3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.368183 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b17621-47ba-4f4e-8d47-cac76ad061c3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.372994 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbe07d-f57c-49bc-9865-508202f3b1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.373591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbe07d-f57c-49bc-9865-508202f3b1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.373872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbe07d-f57c-49bc-9865-508202f3b1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.374758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbe07d-f57c-49bc-9865-508202f3b1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.375415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbe07d-f57c-49bc-9865-508202f3b1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.377034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbe07d-f57c-49bc-9865-508202f3b1e8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.388909 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1000/2003] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:17.393223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896125ca-a21e-4305-910e-20a94325cc9e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.404151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896125ca-a21e-4305-910e-20a94325cc9e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.409872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f7fd8-5d13-459d-a368-3a3a15f08919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.410532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f7fd8-5d13-459d-a368-3a3a15f08919 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.410834 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f7fd8-5d13-459d-a368-3a3a15f08919 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.411171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f7fd8-5d13-459d-a368-3a3a15f08919 admin admin] PATH_INFO: `/v3/projects/a9515366ed6b466d8bdaea3e4b4f3d52/users/36216692286b44ca82602920caf4a30a/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.426693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f7fd8-5d13-459d-a368-3a3a15f08919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9515366ed6b466d8bdaea3e4b4f3d52, user_id=36216692286b44ca82602920caf4a30a, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.429450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f7fd8-5d13-459d-a368-3a3a15f08919 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.443499 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1004/2004] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:54:17 2022] PUT /identity/v3/projects/a9515366ed6b466d8bdaea3e4b4f3d52/users/36216692286b44ca82602920caf4a30a/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:17.454844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796358b7-c359-46c9-842e-da5be635e162 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.455189 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796358b7-c359-46c9-842e-da5be635e162 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.455487 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796358b7-c359-46c9-842e-da5be635e162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.484050 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-796358b7-c359-46c9-842e-da5be635e162 None None] MFA Rules not processed for user `36216692286b44ca82602920caf4a30a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:17.485303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-796358b7-c359-46c9-842e-da5be635e162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:17.561842 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1001/2005] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:17.566939 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f7fd8-5d13-459d-a368-3a3a15f08919 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.631759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f7fd8-5d13-459d-a368-3a3a15f08919 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.637809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fc472-94f5-4679-a9e8-a9fa9adca9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.638444 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fc472-94f5-4679-a9e8-a9fa9adca9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.638753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fc472-94f5-4679-a9e8-a9fa9adca9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.639026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fc472-94f5-4679-a9e8-a9fa9adca9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.639660 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fc472-94f5-4679-a9e8-a9fa9adca9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.641570 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fc472-94f5-4679-a9e8-a9fa9adca9d4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.672681 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1005/2006] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:54:17 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:54:17.677090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796358b7-c359-46c9-842e-da5be635e162 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.687851 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796358b7-c359-46c9-842e-da5be635e162 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.749197 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30ffe1-2daf-493d-9678-2e9a0032120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.749930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30ffe1-2daf-493d-9678-2e9a0032120e admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.750297 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30ffe1-2daf-493d-9678-2e9a0032120e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.750446 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30ffe1-2daf-493d-9678-2e9a0032120e admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.751123 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30ffe1-2daf-493d-9678-2e9a0032120e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.753134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30ffe1-2daf-493d-9678-2e9a0032120e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.763594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:17.776548 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1002/2007] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:54:17 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:54:17.781358 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fc472-94f5-4679-a9e8-a9fa9adca9d4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.791817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fc472-94f5-4679-a9e8-a9fa9adca9d4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.796893 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35527b5d-2740-43d1-ba93-a90388fda390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.797499 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35527b5d-2740-43d1-ba93-a90388fda390 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.797798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35527b5d-2740-43d1-ba93-a90388fda390 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.798071 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35527b5d-2740-43d1-ba93-a90388fda390 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.798798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35527b5d-2740-43d1-ba93-a90388fda390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.800404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35527b5d-2740-43d1-ba93-a90388fda390 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.811324 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1006/2008] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:17.815790 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30ffe1-2daf-493d-9678-2e9a0032120e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.826483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30ffe1-2daf-493d-9678-2e9a0032120e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.832334 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e588e7-1fe1-4016-8856-1f848a3dc2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.833003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e588e7-1fe1-4016-8856-1f848a3dc2af admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.833265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e588e7-1fe1-4016-8856-1f848a3dc2af admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.833534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e588e7-1fe1-4016-8856-1f848a3dc2af admin admin] PATH_INFO: `/v3/projects/8eceb482114d476fae768c361534b5db/users/4d25fbb74a624842bdefccad022ae963/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.849099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e588e7-1fe1-4016-8856-1f848a3dc2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eceb482114d476fae768c361534b5db, user_id=4d25fbb74a624842bdefccad022ae963, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.851807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e588e7-1fe1-4016-8856-1f848a3dc2af admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.865491 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1003/2009] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:54:17 2022] PUT /identity/v3/projects/8eceb482114d476fae768c361534b5db/users/4d25fbb74a624842bdefccad022ae963/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:17.869548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35527b5d-2740-43d1-ba93-a90388fda390 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.879789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35527b5d-2740-43d1-ba93-a90388fda390 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.936252 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35c17-d18e-48b4-b3ab-1ddbdecde898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.936873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35c17-d18e-48b4-b3ab-1ddbdecde898 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.937152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35c17-d18e-48b4-b3ab-1ddbdecde898 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.937425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35c17-d18e-48b4-b3ab-1ddbdecde898 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.938150 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f35c17-d18e-48b4-b3ab-1ddbdecde898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.939789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f35c17-d18e-48b4-b3ab-1ddbdecde898 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.950625 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1007/2010] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:17.954828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e588e7-1fe1-4016-8856-1f848a3dc2af admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:17.965977 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e588e7-1fe1-4016-8856-1f848a3dc2af admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:17.970684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f8405-515c-453f-9b36-305ce3e3024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:17.971385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f8405-515c-453f-9b36-305ce3e3024b admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.971737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f8405-515c-453f-9b36-305ce3e3024b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:17.971938 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f8405-515c-453f-9b36-305ce3e3024b admin admin] PATH_INFO: `/v3/projects/8eceb482114d476fae768c361534b5db/users/4d25fbb74a624842bdefccad022ae963/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:17.974199 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f8405-515c-453f-9b36-305ce3e3024b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eceb482114d476fae768c361534b5db, user_id=4d25fbb74a624842bdefccad022ae963, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:17.977006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f8405-515c-453f-9b36-305ce3e3024b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:17.988527 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1004/2011] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:54:17 2022] PUT /identity/v3/projects/8eceb482114d476fae768c361534b5db/users/4d25fbb74a624842bdefccad022ae963/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:17.999350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e3fd8-98c0-4b8b-899a-3d58ea5e747f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:17.999693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e3fd8-98c0-4b8b-899a-3d58ea5e747f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:18.000052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e3fd8-98c0-4b8b-899a-3d58ea5e747f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:18.028892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-347e3fd8-98c0-4b8b-899a-3d58ea5e747f None None] MFA Rules not processed for user `4d25fbb74a624842bdefccad022ae963`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:18.039156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-347e3fd8-98c0-4b8b-899a-3d58ea5e747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:18.049540 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:18.113460 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1008/2012] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:54:18.127730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f8405-515c-453f-9b36-305ce3e3024b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:18.129580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-227f8405-515c-453f-9b36-305ce3e3024b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:18.195058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f8405-515c-453f-9b36-305ce3e3024b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:18.199020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f05f4-32b5-47df-b39f-c21d0cd9e8db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:18.199698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f05f4-32b5-47df-b39f-c21d0cd9e8db service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:18.200013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f05f4-32b5-47df-b39f-c21d0cd9e8db service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:18.200284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f05f4-32b5-47df-b39f-c21d0cd9e8db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:18.209646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f05f4-32b5-47df-b39f-c21d0cd9e8db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:18.211404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f05f4-32b5-47df-b39f-c21d0cd9e8db service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:18.220986 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1005/2013] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:18.302531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e3fd8-98c0-4b8b-899a-3d58ea5e747f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:18.313420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e3fd8-98c0-4b8b-899a-3d58ea5e747f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:18.317332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846d18-6d3f-44aa-9a22-ce83d4e9c660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:18.317967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846d18-6d3f-44aa-9a22-ce83d4e9c660 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:18.318239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846d18-6d3f-44aa-9a22-ce83d4e9c660 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:18.318505 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846d18-6d3f-44aa-9a22-ce83d4e9c660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:18.376483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf846d18-6d3f-44aa-9a22-ce83d4e9c660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:18.378033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf846d18-6d3f-44aa-9a22-ce83d4e9c660 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:18.387169 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1009/2014] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:54:25.268363 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f05f4-32b5-47df-b39f-c21d0cd9e8db service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:25.332171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f05f4-32b5-47df-b39f-c21d0cd9e8db service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:25.337042 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:25.337672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:25.337933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:25.338205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] PATH_INFO: `/v3/users/36216692286b44ca82602920caf4a30a` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:25.339338 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36216692286b44ca82602920caf4a30a)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:25.341255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:25.391516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36216692286b44ca82602920caf4a30a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:25.392179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36216692286b44ca82602920caf4a30a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:25.399617 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36216692286b44ca82602920caf4a30a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:25.402433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:25.408293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36216692286b44ca82602920caf4a30a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:25.422857 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36216692286b44ca82602920caf4a30a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:25.428898 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1006/2015] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:54:25 2022] DELETE /identity/v3/users/36216692286b44ca82602920caf4a30a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:25.433519 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c634bb-bf59-4d32-996d-c009923835ea None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:25.434066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c634bb-bf59-4d32-996d-c009923835ea None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:25.434286 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c634bb-bf59-4d32-996d-c009923835ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:25.453274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-88c634bb-bf59-4d32-996d-c009923835ea None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:25.454492 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-88c634bb-bf59-4d32-996d-c009923835ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:25.519227 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1010/2016] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:25.525806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:25.527298 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:25.595852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868ddf7-7422-42ac-9eba-2a71a93c63f3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:25.601267 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016ffc3-9d13-4472-a5bc-47a614e34162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:25.601877 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016ffc3-9d13-4472-a5bc-47a614e34162 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:25.602152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016ffc3-9d13-4472-a5bc-47a614e34162 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:25.602421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016ffc3-9d13-4472-a5bc-47a614e34162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:25.611907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016ffc3-9d13-4472-a5bc-47a614e34162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:25.613321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016ffc3-9d13-4472-a5bc-47a614e34162 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:25.622706 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1007/2017] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:25.991121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c634bb-bf59-4d32-996d-c009923835ea None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:25.992540 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-88c634bb-bf59-4d32-996d-c009923835ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:26.013763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c634bb-bf59-4d32-996d-c009923835ea None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:26.020394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:26.021170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:26.021451 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:26.021738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] PATH_INFO: `/v3/projects/a9515366ed6b466d8bdaea3e4b4f3d52` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:26.022873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9515366ed6b466d8bdaea3e4b4f3d52)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:26.024787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:26.080065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:26.082408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9515366ed6b466d8bdaea3e4b4f3d52', 'request_id': 'req-ba1c259b-ef45-4604-98e9-5fdef5d84150'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.090206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9515366ed6b466d8bdaea3e4b4f3d52', 'request_id': 'req-ba1c259b-ef45-4604-98e9-5fdef5d84150'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.096582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9515366ed6b466d8bdaea3e4b4f3d52', 'request_id': 'req-ba1c259b-ef45-4604-98e9-5fdef5d84150'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.109588 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] The token cache is being invalidate because project a9515366ed6b466d8bdaea3e4b4f3d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:54:26.109891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.110515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.112154 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1011/2018] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:54:25 2022] DELETE /identity/v3/projects/a9515366ed6b466d8bdaea3e4b4f3d52 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:54:26.116256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016ffc3-9d13-4472-a5bc-47a614e34162 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:26.126783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016ffc3-9d13-4472-a5bc-47a614e34162 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:26.199577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:26.200894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:26.201233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:26.202828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] PATH_INFO: `/v3/users/4d25fbb74a624842bdefccad022ae963` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:26.203554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d25fbb74a624842bdefccad022ae963)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:26.205860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:26.281304 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d25fbb74a624842bdefccad022ae963'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.283563 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d25fbb74a624842bdefccad022ae963'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.292617 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d25fbb74a624842bdefccad022ae963'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.302132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d25fbb74a624842bdefccad022ae963'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.316082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdc37143-42e7-4378-9401-3d66798bf452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d25fbb74a624842bdefccad022ae963'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.320754 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:26.330952 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1008/2019] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:54:26 2022] DELETE /identity/v3/users/4d25fbb74a624842bdefccad022ae963 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:26.648930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:26.650571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:26.672654 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c259b-ef45-4604-98e9-5fdef5d84150 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:26.732468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:26.733116 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:26.733522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:26.733798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] PATH_INFO: `/v3/projects/8eceb482114d476fae768c361534b5db` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:26.734942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eceb482114d476fae768c361534b5db)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:26.736897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:26.787175 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eceb482114d476fae768c361534b5db', 'request_id': 'req-e060cd20-69d1-4716-8288-f1ccfc4ab46a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.793943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eceb482114d476fae768c361534b5db', 'request_id': 'req-e060cd20-69d1-4716-8288-f1ccfc4ab46a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.802078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eceb482114d476fae768c361534b5db', 'request_id': 'req-e060cd20-69d1-4716-8288-f1ccfc4ab46a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.815614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] The token cache is being invalidate because project 8eceb482114d476fae768c361534b5db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:54:26.815820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.816477 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e060cd20-69d1-4716-8288-f1ccfc4ab46a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:26.821573 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1012/2020] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:54:26 2022] DELETE /identity/v3/projects/8eceb482114d476fae768c361534b5db => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:26.823124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804c177-4f82-4fbf-b53a-e179cdd3c4d8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:26.823413 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804c177-4f82-4fbf-b53a-e179cdd3c4d8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:26.823687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804c177-4f82-4fbf-b53a-e179cdd3c4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:26.846114 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8804c177-4f82-4fbf-b53a-e179cdd3c4d8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:26.847721 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8804c177-4f82-4fbf-b53a-e179cdd3c4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:26.914727 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1009/2021] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:54:26.923860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84cf66-1f52-4aca-b361-2aceaa167dbb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:26.924554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84cf66-1f52-4aca-b361-2aceaa167dbb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:26.925025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84cf66-1f52-4aca-b361-2aceaa167dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:26.949188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ee84cf66-1f52-4aca-b361-2aceaa167dbb None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:26.950474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ee84cf66-1f52-4aca-b361-2aceaa167dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:26.959529 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1013/2022] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:26.963808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804c177-4f82-4fbf-b53a-e179cdd3c4d8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:26.974228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804c177-4f82-4fbf-b53a-e179cdd3c4d8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:26.979254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c1520-8e18-4dd6-9d03-1544b195718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:26.979873 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c1520-8e18-4dd6-9d03-1544b195718e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:26.980152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c1520-8e18-4dd6-9d03-1544b195718e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:26.980427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c1520-8e18-4dd6-9d03-1544b195718e admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:26.981100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c1520-8e18-4dd6-9d03-1544b195718e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:54:26.981401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c1520-8e18-4dd6-9d03-1544b195718e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:26.983027 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c1520-8e18-4dd6-9d03-1544b195718e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:26.995497 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1010/2023] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:54:26.999534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84cf66-1f52-4aca-b361-2aceaa167dbb None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.010852 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84cf66-1f52-4aca-b361-2aceaa167dbb None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.016074 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e8875-bc90-495c-81fc-4355eb04016f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.016695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e8875-bc90-495c-81fc-4355eb04016f admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.016969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e8875-bc90-495c-81fc-4355eb04016f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.017238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e8875-bc90-495c-81fc-4355eb04016f admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.018091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e8875-bc90-495c-81fc-4355eb04016f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.020178 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e8875-bc90-495c-81fc-4355eb04016f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.057755 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1014/2024] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:54:26 2022] POST /identity/v3/projects => generated 409 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:54:27.058455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c1520-8e18-4dd6-9d03-1544b195718e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.069467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c1520-8e18-4dd6-9d03-1544b195718e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.137910 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075689-ae50-4e62-ab57-a840830b8273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.138533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075689-ae50-4e62-ab57-a840830b8273 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.139395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075689-ae50-4e62-ab57-a840830b8273 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.139395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075689-ae50-4e62-ab57-a840830b8273 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.139780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99075689-ae50-4e62-ab57-a840830b8273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.141688 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99075689-ae50-4e62-ab57-a840830b8273 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.172618 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1011/2025] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:54:27 2022] POST /identity/v3/users => generated 305 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:54:27.173816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e8875-bc90-495c-81fc-4355eb04016f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.186217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e8875-bc90-495c-81fc-4355eb04016f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.191224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11866065-1666-4ce4-9c2c-ddbf3f896660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.191871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11866065-1666-4ce4-9c2c-ddbf3f896660 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.192151 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11866065-1666-4ce4-9c2c-ddbf3f896660 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.192425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11866065-1666-4ce4-9c2c-ddbf3f896660 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.193075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11866065-1666-4ce4-9c2c-ddbf3f896660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.194693 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11866065-1666-4ce4-9c2c-ddbf3f896660 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.205457 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1015/2026] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:27.209963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075689-ae50-4e62-ab57-a840830b8273 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.227404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075689-ae50-4e62-ab57-a840830b8273 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.232400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed66abc-b320-428c-8617-abe811ec3fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.233120 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed66abc-b320-428c-8617-abe811ec3fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.233497 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed66abc-b320-428c-8617-abe811ec3fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.233865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed66abc-b320-428c-8617-abe811ec3fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.234614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed66abc-b320-428c-8617-abe811ec3fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.236351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed66abc-b320-428c-8617-abe811ec3fe8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.249339 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1012/2027] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:54:27.253443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11866065-1666-4ce4-9c2c-ddbf3f896660 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.264990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11866065-1666-4ce4-9c2c-ddbf3f896660 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.275725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356c5a4-c46e-4dfc-85d9-0b948b8de436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.276996 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356c5a4-c46e-4dfc-85d9-0b948b8de436 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.277278 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356c5a4-c46e-4dfc-85d9-0b948b8de436 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.277548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356c5a4-c46e-4dfc-85d9-0b948b8de436 admin admin] PATH_INFO: `/v3/projects/23b62ade2ed44b9c9a258c802a3be18c/users/f31790ef73c34a3b9f5901d192133f7a/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.300566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356c5a4-c46e-4dfc-85d9-0b948b8de436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b62ade2ed44b9c9a258c802a3be18c, user_id=f31790ef73c34a3b9f5901d192133f7a, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.303457 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356c5a4-c46e-4dfc-85d9-0b948b8de436 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.313250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:27.332810 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1016/2028] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:54:27 2022] PUT /identity/v3/projects/23b62ade2ed44b9c9a258c802a3be18c/users/f31790ef73c34a3b9f5901d192133f7a/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:54:27.334315 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981215b3-e68c-4167-bca1-394649da9669 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.334832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981215b3-e68c-4167-bca1-394649da9669 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.334939 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981215b3-e68c-4167-bca1-394649da9669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.364979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-981215b3-e68c-4167-bca1-394649da9669 None None] MFA Rules not processed for user `f31790ef73c34a3b9f5901d192133f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:27.366201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-981215b3-e68c-4167-bca1-394649da9669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:27.424970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:27.440876 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1013/2029] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:27 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:27.445104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356c5a4-c46e-4dfc-85d9-0b948b8de436 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.514429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356c5a4-c46e-4dfc-85d9-0b948b8de436 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.520097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c461d3-5d28-40d8-ac85-614893ddbba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.520789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c461d3-5d28-40d8-ac85-614893ddbba8 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.520993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c461d3-5d28-40d8-ac85-614893ddbba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.521270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c461d3-5d28-40d8-ac85-614893ddbba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.522123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c461d3-5d28-40d8-ac85-614893ddbba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.524227 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c461d3-5d28-40d8-ac85-614893ddbba8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.554293 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1017/2030] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:54:27 2022] POST /identity/v3/projects => generated 407 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:54:27.558477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981215b3-e68c-4167-bca1-394649da9669 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.569213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981215b3-e68c-4167-bca1-394649da9669 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.626046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75902630-5cd3-4606-9f09-aad80f065bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.626664 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75902630-5cd3-4606-9f09-aad80f065bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.626972 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75902630-5cd3-4606-9f09-aad80f065bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.627274 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75902630-5cd3-4606-9f09-aad80f065bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.627903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75902630-5cd3-4606-9f09-aad80f065bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.629825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75902630-5cd3-4606-9f09-aad80f065bb8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.655010 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1014/2031] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:54:27 2022] POST /identity/v3/users => generated 310 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:54:27.656538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c461d3-5d28-40d8-ac85-614893ddbba8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.667965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c461d3-5d28-40d8-ac85-614893ddbba8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.673469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a00112-2766-471a-b50d-bf329d98c6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.674288 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a00112-2766-471a-b50d-bf329d98c6ec admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.674679 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a00112-2766-471a-b50d-bf329d98c6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.674845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a00112-2766-471a-b50d-bf329d98c6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.675486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a00112-2766-471a-b50d-bf329d98c6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.677084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a00112-2766-471a-b50d-bf329d98c6ec admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.694121 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1018/2032] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:27.695983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75902630-5cd3-4606-9f09-aad80f065bb8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.706976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75902630-5cd3-4606-9f09-aad80f065bb8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.711571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3e884-9a3e-4967-8670-68011d9a67ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.712225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3e884-9a3e-4967-8670-68011d9a67ab admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.712505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3e884-9a3e-4967-8670-68011d9a67ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.712779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3e884-9a3e-4967-8670-68011d9a67ab admin admin] PATH_INFO: `/v3/projects/3dc4a06dd3004b2b86a6240ce5afbca9/users/41c22b7da9e74d289a50f1e2632de6e5/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.728246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3e884-9a3e-4967-8670-68011d9a67ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc4a06dd3004b2b86a6240ce5afbca9, user_id=41c22b7da9e74d289a50f1e2632de6e5, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.731205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3e884-9a3e-4967-8670-68011d9a67ab admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.745077 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1015/2033] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:54:27 2022] PUT /identity/v3/projects/3dc4a06dd3004b2b86a6240ce5afbca9/users/41c22b7da9e74d289a50f1e2632de6e5/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:54:27.749470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a00112-2766-471a-b50d-bf329d98c6ec admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.760067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a00112-2766-471a-b50d-bf329d98c6ec admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.818057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee8a9-5e0d-4429-beff-0204b51cfa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.818734 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee8a9-5e0d-4429-beff-0204b51cfa07 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.818968 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee8a9-5e0d-4429-beff-0204b51cfa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.819235 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee8a9-5e0d-4429-beff-0204b51cfa07 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.819879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ee8a9-5e0d-4429-beff-0204b51cfa07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.821462 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ee8a9-5e0d-4429-beff-0204b51cfa07 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.831744 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1019/2034] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:27.836120 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3e884-9a3e-4967-8670-68011d9a67ab admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:27.846941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3e884-9a3e-4967-8670-68011d9a67ab admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:27.851969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10618f-034e-4689-b380-0329f47cb300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:27.852697 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10618f-034e-4689-b380-0329f47cb300 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.852980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10618f-034e-4689-b380-0329f47cb300 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.853253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10618f-034e-4689-b380-0329f47cb300 admin admin] PATH_INFO: `/v3/projects/3dc4a06dd3004b2b86a6240ce5afbca9/users/41c22b7da9e74d289a50f1e2632de6e5/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.855364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10618f-034e-4689-b380-0329f47cb300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc4a06dd3004b2b86a6240ce5afbca9, user_id=41c22b7da9e74d289a50f1e2632de6e5, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:27.858213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10618f-034e-4689-b380-0329f47cb300 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:27.868862 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1016/2035] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:54:27 2022] PUT /identity/v3/projects/3dc4a06dd3004b2b86a6240ce5afbca9/users/41c22b7da9e74d289a50f1e2632de6e5/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:27.879645 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1b258-819e-450e-bd5f-ebc3de3b98e3 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:27.879963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1b258-819e-450e-bd5f-ebc3de3b98e3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:27.880343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1b258-819e-450e-bd5f-ebc3de3b98e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:27.909910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-15a1b258-819e-450e-bd5f-ebc3de3b98e3 None None] MFA Rules not processed for user `41c22b7da9e74d289a50f1e2632de6e5`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:27.911162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-15a1b258-819e-450e-bd5f-ebc3de3b98e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:27.981710 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:27.992209 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1020/2036] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:27 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:28.004208 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10618f-034e-4689-b380-0329f47cb300 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:28.006069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6b10618f-034e-4689-b380-0329f47cb300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:28.067231 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10618f-034e-4689-b380-0329f47cb300 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:28.071293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076452f-521e-4342-abbe-53e03d22bf56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:28.072185 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076452f-521e-4342-abbe-53e03d22bf56 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:28.072419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076452f-521e-4342-abbe-53e03d22bf56 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:28.072703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076452f-521e-4342-abbe-53e03d22bf56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:28.082931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076452f-521e-4342-abbe-53e03d22bf56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:28.084422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076452f-521e-4342-abbe-53e03d22bf56 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:28.094373 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1017/2037] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:28 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:28.252113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1b258-819e-450e-bd5f-ebc3de3b98e3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:28.261789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1b258-819e-450e-bd5f-ebc3de3b98e3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:28.265337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c1488-a611-44c6-8db3-fa2bef814cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:28.265946 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c1488-a611-44c6-8db3-fa2bef814cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:28.266228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c1488-a611-44c6-8db3-fa2bef814cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:28.266549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c1488-a611-44c6-8db3-fa2bef814cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:28.328326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c1488-a611-44c6-8db3-fa2bef814cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:28.329775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c1488-a611-44c6-8db3-fa2bef814cd3 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:28.338944 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1021/2038] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:28 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:29.027886 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96b0c4-f673-42c9-b484-3db59200f346 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:29.028415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96b0c4-f673-42c9-b484-3db59200f346 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:29.028415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96b0c4-f673-42c9-b484-3db59200f346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:29.048278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6e96b0c4-f673-42c9-b484-3db59200f346 None None] MFA Rules not processed for user `712e5cf4cc004918817ab1802b9c69d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:29.049466 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6e96b0c4-f673-42c9-b484-3db59200f346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:29.099029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:29.111679 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1018/2039] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:33.707984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c1488-a611-44c6-8db3-fa2bef814cd3 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:33.772096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c1488-a611-44c6-8db3-fa2bef814cd3 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:33.777286 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:33.777827 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:33.778103 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:33.778373 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] PATH_INFO: `/v3/users/f31790ef73c34a3b9f5901d192133f7a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:33.779578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31790ef73c34a3b9f5901d192133f7a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:33.781582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:33.830209 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31790ef73c34a3b9f5901d192133f7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:33.837088 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31790ef73c34a3b9f5901d192133f7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:33.837817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31790ef73c34a3b9f5901d192133f7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:33.845246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31790ef73c34a3b9f5901d192133f7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:33.858278 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31790ef73c34a3b9f5901d192133f7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:33.864453 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1022/2040] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:54:33 2022] DELETE /identity/v3/users/f31790ef73c34a3b9f5901d192133f7a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:33.869226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45f525-d1e9-4ba7-b229-a4d7cf01833f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:33.869725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45f525-d1e9-4ba7-b229-a4d7cf01833f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:33.869725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45f525-d1e9-4ba7-b229-a4d7cf01833f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:33.890897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3c45f525-d1e9-4ba7-b229-a4d7cf01833f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:33.892264 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3c45f525-d1e9-4ba7-b229-a4d7cf01833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:33.954276 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1019/2041] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:33.964414 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:33.966053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:33.982370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:34.028134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e17d3-a274-4199-b872-8aa0b69fe3d7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:34.032420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216e2cd-c3b3-467d-a00d-a4293e978c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:34.033035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216e2cd-c3b3-467d-a00d-a4293e978c3a service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:34.033308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216e2cd-c3b3-467d-a00d-a4293e978c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:34.033586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216e2cd-c3b3-467d-a00d-a4293e978c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:34.043951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216e2cd-c3b3-467d-a00d-a4293e978c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:34.045354 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216e2cd-c3b3-467d-a00d-a4293e978c3a service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:34.055750 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1023/2042] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:34.337077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45f525-d1e9-4ba7-b229-a4d7cf01833f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:34.341899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3c45f525-d1e9-4ba7-b229-a4d7cf01833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:34.352049 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45f525-d1e9-4ba7-b229-a4d7cf01833f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:34.358101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:34.358845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:34.359134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:34.359409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] PATH_INFO: `/v3/projects/23b62ade2ed44b9c9a258c802a3be18c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:34.360662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b62ade2ed44b9c9a258c802a3be18c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:34.362672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:34.411364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b62ade2ed44b9c9a258c802a3be18c', 'request_id': 'req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.416954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b62ade2ed44b9c9a258c802a3be18c', 'request_id': 'req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.424037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b62ade2ed44b9c9a258c802a3be18c', 'request_id': 'req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.437832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] The token cache is being invalidate because project 23b62ade2ed44b9c9a258c802a3be18c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:54:34.438123 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.438746 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.440412 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1020/2043] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:54:34 2022] DELETE /identity/v3/projects/23b62ade2ed44b9c9a258c802a3be18c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:34.444389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216e2cd-c3b3-467d-a00d-a4293e978c3a service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:34.453983 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216e2cd-c3b3-467d-a00d-a4293e978c3a service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:34.512030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:34.512649 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:34.512925 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:34.513195 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] PATH_INFO: `/v3/users/41c22b7da9e74d289a50f1e2632de6e5` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:34.514342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c22b7da9e74d289a50f1e2632de6e5)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:34.516277 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:34.566002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c22b7da9e74d289a50f1e2632de6e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.572498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c22b7da9e74d289a50f1e2632de6e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.573140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c22b7da9e74d289a50f1e2632de6e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.581387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c22b7da9e74d289a50f1e2632de6e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.593538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f0b1cda-936e-4151-a42b-cced0fd6f985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c22b7da9e74d289a50f1e2632de6e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:34.599877 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1024/2044] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:54:34 2022] DELETE /identity/v3/users/41c22b7da9e74d289a50f1e2632de6e5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:34.883294 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:34.884874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:34.902355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a0d2a-dc69-47df-bd0f-c8d3040d5e9d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:34.960952 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:34.961603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:34.961880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:34.962155 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] PATH_INFO: `/v3/projects/3dc4a06dd3004b2b86a6240ce5afbca9` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:34.963351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc4a06dd3004b2b86a6240ce5afbca9)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:34.965242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:34.988314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:35.016059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc4a06dd3004b2b86a6240ce5afbca9', 'request_id': 'req-273ac647-d7f3-4f26-9ab7-e084301fd533'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:35.022649 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc4a06dd3004b2b86a6240ce5afbca9', 'request_id': 'req-273ac647-d7f3-4f26-9ab7-e084301fd533'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:35.029012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc4a06dd3004b2b86a6240ce5afbca9', 'request_id': 'req-273ac647-d7f3-4f26-9ab7-e084301fd533'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:35.042966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] The token cache is being invalidate because project 3dc4a06dd3004b2b86a6240ce5afbca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:54:35.043465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:35.044309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-273ac647-d7f3-4f26-9ab7-e084301fd533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:54:35.046259 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1021/2045] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:54:34 2022] DELETE /identity/v3/projects/3dc4a06dd3004b2b86a6240ce5afbca9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:35.051221 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67eac7d-a694-48f9-859a-71fb726d704f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.051717 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67eac7d-a694-48f9-859a-71fb726d704f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.051816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67eac7d-a694-48f9-859a-71fb726d704f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.071887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f67eac7d-a694-48f9-859a-71fb726d704f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:35.073276 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f67eac7d-a694-48f9-859a-71fb726d704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:35.104206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:35.134904 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1025/2046] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:35.143909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d774cc4-0f25-4ec9-b52e-784ee620b925 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.144258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d774cc4-0f25-4ec9-b52e-784ee620b925 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.144551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d774cc4-0f25-4ec9-b52e-784ee620b925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.163932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1d774cc4-0f25-4ec9-b52e-784ee620b925 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:35.165125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1d774cc4-0f25-4ec9-b52e-784ee620b925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:35.173145 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1022/2047] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:35.177483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67eac7d-a694-48f9-859a-71fb726d704f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:35.187667 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67eac7d-a694-48f9-859a-71fb726d704f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:35.192428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35081be6-86d9-4d17-9696-8729b3dcdf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:35.193194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35081be6-86d9-4d17-9696-8729b3dcdf1e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.193472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35081be6-86d9-4d17-9696-8729b3dcdf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.193744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35081be6-86d9-4d17-9696-8729b3dcdf1e admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.194397 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35081be6-86d9-4d17-9696-8729b3dcdf1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:54:35.194695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35081be6-86d9-4d17-9696-8729b3dcdf1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:35.196452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35081be6-86d9-4d17-9696-8729b3dcdf1e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:35.208644 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1026/2048] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:54:35.213141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d774cc4-0f25-4ec9-b52e-784ee620b925 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:35.222508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d774cc4-0f25-4ec9-b52e-784ee620b925 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:35.227262 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee37739-0f24-4bbd-b66d-5776a43ea1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:35.227927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee37739-0f24-4bbd-b66d-5776a43ea1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.228266 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee37739-0f24-4bbd-b66d-5776a43ea1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.228539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee37739-0f24-4bbd-b66d-5776a43ea1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.229186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee37739-0f24-4bbd-b66d-5776a43ea1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:35.231095 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee37739-0f24-4bbd-b66d-5776a43ea1d2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:35.263064 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1023/2049] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:54:35 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:54:35.264196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35081be6-86d9-4d17-9696-8729b3dcdf1e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:35.274737 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35081be6-86d9-4d17-9696-8729b3dcdf1e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:35.331865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf1033-1309-4693-a526-9c0c6e91f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:35.332382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf1033-1309-4693-a526-9c0c6e91f3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.332655 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf1033-1309-4693-a526-9c0c6e91f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.332921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf1033-1309-4693-a526-9c0c6e91f3f6 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.333551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf1033-1309-4693-a526-9c0c6e91f3f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:35.335498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf1033-1309-4693-a526-9c0c6e91f3f6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:35.359619 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1027/2050] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:54:35 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:54:35.363908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee37739-0f24-4bbd-b66d-5776a43ea1d2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:35.374107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee37739-0f24-4bbd-b66d-5776a43ea1d2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:35.379158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6a155-bfb9-44f8-9d22-149355249441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:35.379767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6a155-bfb9-44f8-9d22-149355249441 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.380040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6a155-bfb9-44f8-9d22-149355249441 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.380307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6a155-bfb9-44f8-9d22-149355249441 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.380950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6a155-bfb9-44f8-9d22-149355249441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:35.382538 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6a155-bfb9-44f8-9d22-149355249441 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:35.392728 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1024/2051] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:35.397113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf1033-1309-4693-a526-9c0c6e91f3f6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:35.408293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf1033-1309-4693-a526-9c0c6e91f3f6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:35.413498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabfe34-f6ce-4c55-a4d5-8aa062a305c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:35.414232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabfe34-f6ce-4c55-a4d5-8aa062a305c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.414520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabfe34-f6ce-4c55-a4d5-8aa062a305c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.414862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabfe34-f6ce-4c55-a4d5-8aa062a305c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.415513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabfe34-f6ce-4c55-a4d5-8aa062a305c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:35.417128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabfe34-f6ce-4c55-a4d5-8aa062a305c5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:35.428934 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1028/2052] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:35.433346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6a155-bfb9-44f8-9d22-149355249441 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:35.443428 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6a155-bfb9-44f8-9d22-149355249441 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:35.448811 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5ebfb-bba9-4b89-bcd5-63b060fe56dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:35.449456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5ebfb-bba9-4b89-bcd5-63b060fe56dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.449738 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5ebfb-bba9-4b89-bcd5-63b060fe56dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.449986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5ebfb-bba9-4b89-bcd5-63b060fe56dc admin admin] PATH_INFO: `/v3/projects/90625756dd824771a6221a0b919afe81/users/f4806783c2df4220b7bc37090ba2b40d/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.465204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5ebfb-bba9-4b89-bcd5-63b060fe56dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=90625756dd824771a6221a0b919afe81, user_id=f4806783c2df4220b7bc37090ba2b40d, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:35.467868 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5ebfb-bba9-4b89-bcd5-63b060fe56dc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:35.481729 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1025/2053] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:54:35 2022] PUT /identity/v3/projects/90625756dd824771a6221a0b919afe81/users/f4806783c2df4220b7bc37090ba2b40d/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:35.487008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa284f7-37d4-4e4f-9987-ccff2a0d610d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.487319 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa284f7-37d4-4e4f-9987-ccff2a0d610d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.487588 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa284f7-37d4-4e4f-9987-ccff2a0d610d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.508981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4aa284f7-37d4-4e4f-9987-ccff2a0d610d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:35.510186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4aa284f7-37d4-4e4f-9987-ccff2a0d610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:35.568220 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1029/2054] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:35.578176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5ebfb-bba9-4b89-bcd5-63b060fe56dc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:35.580046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-57d5ebfb-bba9-4b89-bcd5-63b060fe56dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:35.641181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5ebfb-bba9-4b89-bcd5-63b060fe56dc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:35.645494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5505e-2749-49d3-918c-de2742a55430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:35.646108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5505e-2749-49d3-918c-de2742a55430 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:35.646385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5505e-2749-49d3-918c-de2742a55430 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:35.646652 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5505e-2749-49d3-918c-de2742a55430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:35.658906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5505e-2749-49d3-918c-de2742a55430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:35.660523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5505e-2749-49d3-918c-de2742a55430 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:35.672144 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1026/2055] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:36.313447 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a98ada-b95a-4939-8a9d-2543931d6830 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:36.313969 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a98ada-b95a-4939-8a9d-2543931d6830 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:36.314088 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a98ada-b95a-4939-8a9d-2543931d6830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:36.343233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a5a98ada-b95a-4939-8a9d-2543931d6830 None None] MFA Rules not processed for user `f4806783c2df4220b7bc37090ba2b40d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:36.344479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a5a98ada-b95a-4939-8a9d-2543931d6830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:36.377777 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:36.414086 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1030/2056] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:36.418641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5505e-2749-49d3-918c-de2742a55430 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:36.428913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5505e-2749-49d3-918c-de2742a55430 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:36.433507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f3983-7b20-4e91-b7f9-0bca01372cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:36.434103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f3983-7b20-4e91-b7f9-0bca01372cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:36.434381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f3983-7b20-4e91-b7f9-0bca01372cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:36.434648 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f3983-7b20-4e91-b7f9-0bca01372cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:36.435318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f3983-7b20-4e91-b7f9-0bca01372cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:36.437247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f3983-7b20-4e91-b7f9-0bca01372cf5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:36.467067 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1027/2057] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:54:36 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:54:36.471083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a98ada-b95a-4939-8a9d-2543931d6830 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:36.482396 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a98ada-b95a-4939-8a9d-2543931d6830 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:36.539202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb71e99-0f40-470e-a679-50431f02ff3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:36.539818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb71e99-0f40-470e-a679-50431f02ff3e admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:36.540094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb71e99-0f40-470e-a679-50431f02ff3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:36.540366 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb71e99-0f40-470e-a679-50431f02ff3e admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:36.541040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb71e99-0f40-470e-a679-50431f02ff3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:36.542992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb71e99-0f40-470e-a679-50431f02ff3e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:36.568910 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1031/2058] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:54:36 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:54:36.573535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f3983-7b20-4e91-b7f9-0bca01372cf5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:36.584642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f3983-7b20-4e91-b7f9-0bca01372cf5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:36.590507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee52de-6f70-4501-8c37-af4c8db01499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:36.591186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee52de-6f70-4501-8c37-af4c8db01499 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:36.591465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee52de-6f70-4501-8c37-af4c8db01499 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:36.591904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee52de-6f70-4501-8c37-af4c8db01499 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:36.592558 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee52de-6f70-4501-8c37-af4c8db01499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:36.594267 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee52de-6f70-4501-8c37-af4c8db01499 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:36.605857 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1028/2059] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:54:36.610040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb71e99-0f40-470e-a679-50431f02ff3e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:36.620302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb71e99-0f40-470e-a679-50431f02ff3e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:36.625134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55995c8-787d-4836-8784-ec7b2409dd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:36.625777 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55995c8-787d-4836-8784-ec7b2409dd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:36.626055 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55995c8-787d-4836-8784-ec7b2409dd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:36.626330 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55995c8-787d-4836-8784-ec7b2409dd90 admin admin] PATH_INFO: `/v3/projects/4fa406c8070341dda5edc52d31a25194/users/15e411d07f6d4c6b929ac84d6e350b42/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:36.643261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55995c8-787d-4836-8784-ec7b2409dd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa406c8070341dda5edc52d31a25194, user_id=15e411d07f6d4c6b929ac84d6e350b42, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:36.646346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55995c8-787d-4836-8784-ec7b2409dd90 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:36.661575 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1032/2060] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:54:36 2022] PUT /identity/v3/projects/4fa406c8070341dda5edc52d31a25194/users/15e411d07f6d4c6b929ac84d6e350b42/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:54:36.665766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee52de-6f70-4501-8c37-af4c8db01499 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:36.680096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee52de-6f70-4501-8c37-af4c8db01499 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:36.691684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:36.750132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834539ef-2129-4263-9ca0-59bec3987bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:36.750862 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834539ef-2129-4263-9ca0-59bec3987bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:36.751210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834539ef-2129-4263-9ca0-59bec3987bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:36.751548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834539ef-2129-4263-9ca0-59bec3987bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:36.752253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834539ef-2129-4263-9ca0-59bec3987bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:36.754134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834539ef-2129-4263-9ca0-59bec3987bb2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:36.766157 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1029/2061] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:54:36.770849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55995c8-787d-4836-8784-ec7b2409dd90 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:36.781970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55995c8-787d-4836-8784-ec7b2409dd90 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:36.787001 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0e346-6d64-4146-bd2b-76e7f4cde07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:36.787514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0e346-6d64-4146-bd2b-76e7f4cde07e admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:36.787787 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0e346-6d64-4146-bd2b-76e7f4cde07e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:36.788067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0e346-6d64-4146-bd2b-76e7f4cde07e admin admin] PATH_INFO: `/v3/projects/4fa406c8070341dda5edc52d31a25194/users/15e411d07f6d4c6b929ac84d6e350b42/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:36.789995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0e346-6d64-4146-bd2b-76e7f4cde07e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa406c8070341dda5edc52d31a25194, user_id=15e411d07f6d4c6b929ac84d6e350b42, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:36.792657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0e346-6d64-4146-bd2b-76e7f4cde07e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:36.804011 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1033/2062] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:54:36 2022] PUT /identity/v3/projects/4fa406c8070341dda5edc52d31a25194/users/15e411d07f6d4c6b929ac84d6e350b42/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:54:37.424636 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965a31-f6a1-4dd6-a37e-2ce6020f9998 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:37.425165 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965a31-f6a1-4dd6-a37e-2ce6020f9998 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:37.425165 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965a31-f6a1-4dd6-a37e-2ce6020f9998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:37.455846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a9965a31-f6a1-4dd6-a37e-2ce6020f9998 None None] MFA Rules not processed for user `15e411d07f6d4c6b929ac84d6e350b42`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:37.457125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a9965a31-f6a1-4dd6-a37e-2ce6020f9998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:37.532090 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1030/2063] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:54:37.543786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d17c-c54c-462f-8e49-14c1c6d3bc13 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:37.544334 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d17c-c54c-462f-8e49-14c1c6d3bc13 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:37.544334 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d17c-c54c-462f-8e49-14c1c6d3bc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:37.565484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-69a8d17c-c54c-462f-8e49-14c1c6d3bc13 None None] MFA Rules not processed for user `f4806783c2df4220b7bc37090ba2b40d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:54:37.566822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-69a8d17c-c54c-462f-8e49-14c1c6d3bc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:37.621363 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1034/2064] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:54:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:54:37.632613 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9965a31-f6a1-4dd6-a37e-2ce6020f9998 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:37.634098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a9965a31-f6a1-4dd6-a37e-2ce6020f9998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:37.710006 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9965a31-f6a1-4dd6-a37e-2ce6020f9998 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:37.717178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c20e-1e57-4e19-ae76-d2d3732b6c38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:37.717783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c20e-1e57-4e19-ae76-d2d3732b6c38 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:37.718062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c20e-1e57-4e19-ae76-d2d3732b6c38 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:37.718356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c20e-1e57-4e19-ae76-d2d3732b6c38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:37.726704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224c20e-1e57-4e19-ae76-d2d3732b6c38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:37.728312 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224c20e-1e57-4e19-ae76-d2d3732b6c38 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:37.737585 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1031/2065] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:54:47.614818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:47.733271 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:54:52.971308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8d17c-c54c-462f-8e49-14c1c6d3bc13 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:54:52.972939 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-69a8d17c-c54c-462f-8e49-14c1c6d3bc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:54:53.029813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8d17c-c54c-462f-8e49-14c1c6d3bc13 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:54:53.035187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d3a99-b694-4d0a-bc9d-ba131f407fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:54:53.035808 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d3a99-b694-4d0a-bc9d-ba131f407fec service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:54:53.036178 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d3a99-b694-4d0a-bc9d-ba131f407fec service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:54:53.036536 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d3a99-b694-4d0a-bc9d-ba131f407fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:54:53.051094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d3a99-b694-4d0a-bc9d-ba131f407fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:54:53.052533 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d3a99-b694-4d0a-bc9d-ba131f407fec service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:54:53.062565 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1035/2066] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:54:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:03.057887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=26 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:03.841899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409358d-7bea-43ff-a137-383c5055e1f7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:03.842437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409358d-7bea-43ff-a137-383c5055e1f7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:03.842437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409358d-7bea-43ff-a137-383c5055e1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:03.866232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4409358d-7bea-43ff-a137-383c5055e1f7 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:03.867476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4409358d-7bea-43ff-a137-383c5055e1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:03.933344 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1032/2067] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:55:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:03.989619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d3a99-b694-4d0a-bc9d-ba131f407fec service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:03.991145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0f2d3a99-b694-4d0a-bc9d-ba131f407fec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:04.077752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d3a99-b694-4d0a-bc9d-ba131f407fec service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:04.083480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2da00-4a39-4e7a-9005-0859726f1904 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:04.084110 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2da00-4a39-4e7a-9005-0859726f1904 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:04.084368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2da00-4a39-4e7a-9005-0859726f1904 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:04.084634 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2da00-4a39-4e7a-9005-0859726f1904 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:04.094627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2da00-4a39-4e7a-9005-0859726f1904 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:04.096275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2da00-4a39-4e7a-9005-0859726f1904 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:04.104819 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1036/2068] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:55:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:55:13.926039 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=25 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:14.101270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=32 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:14.862535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409358d-7bea-43ff-a137-383c5055e1f7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:14.923439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409358d-7bea-43ff-a137-383c5055e1f7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:14.928251 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:14.928844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:14.929117 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:14.929385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] PATH_INFO: `/v3/users/f4806783c2df4220b7bc37090ba2b40d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:14.930491 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4806783c2df4220b7bc37090ba2b40d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:14.932427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:14.980134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4806783c2df4220b7bc37090ba2b40d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:14.980768 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4806783c2df4220b7bc37090ba2b40d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:14.987647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4806783c2df4220b7bc37090ba2b40d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:14.996580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4806783c2df4220b7bc37090ba2b40d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.008627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4806783c2df4220b7bc37090ba2b40d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.014788 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1033/2069] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:14 2022] DELETE /identity/v3/users/f4806783c2df4220b7bc37090ba2b40d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:15.237323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2da00-4a39-4e7a-9005-0859726f1904 service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:15.248526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2da00-4a39-4e7a-9005-0859726f1904 service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:15.311096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:15.311728 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:15.312007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:15.312275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] PATH_INFO: `/v3/projects/90625756dd824771a6221a0b919afe81` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:15.313399 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90625756dd824771a6221a0b919afe81)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:15.315345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:15.361993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90625756dd824771a6221a0b919afe81', 'request_id': 'req-2497781b-6885-4f8d-8a91-81356ff208a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.367313 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90625756dd824771a6221a0b919afe81', 'request_id': 'req-2497781b-6885-4f8d-8a91-81356ff208a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.373292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90625756dd824771a6221a0b919afe81', 'request_id': 'req-2497781b-6885-4f8d-8a91-81356ff208a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.386972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] The token cache is being invalidate because project 90625756dd824771a6221a0b919afe81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:15.387237 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.387888 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.389480 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1037/2070] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:15 2022] DELETE /identity/v3/projects/90625756dd824771a6221a0b919afe81 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:15.393689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:15.395262 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:15.408971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961bdd2-f67e-498b-997a-1a680b36ef3f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:15.466801 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:15.467462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:15.467763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:15.468038 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] PATH_INFO: `/v3/users/15e411d07f6d4c6b929ac84d6e350b42` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:15.469194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=15e411d07f6d4c6b929ac84d6e350b42)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:15.471230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:15.493705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:15.521749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15e411d07f6d4c6b929ac84d6e350b42'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.522396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15e411d07f6d4c6b929ac84d6e350b42'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.529853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15e411d07f6d4c6b929ac84d6e350b42'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.538963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15e411d07f6d4c6b929ac84d6e350b42'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.552256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-551e4b2c-b517-4f15-b137-6f82ab8e9a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15e411d07f6d4c6b929ac84d6e350b42'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.558409 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1034/2071] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:15 2022] DELETE /identity/v3/users/15e411d07f6d4c6b929ac84d6e350b42 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:55:15.736571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:15.738848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:15.759422 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497781b-6885-4f8d-8a91-81356ff208a6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:15.824308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:15.824948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:15.825223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:15.825493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] PATH_INFO: `/v3/projects/4fa406c8070341dda5edc52d31a25194` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:15.826649 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa406c8070341dda5edc52d31a25194)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:15.828542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:15.864567 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:15.877140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa406c8070341dda5edc52d31a25194', 'request_id': 'req-cebd28b9-20a8-48de-905b-1e32f57ee10a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.883158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa406c8070341dda5edc52d31a25194', 'request_id': 'req-cebd28b9-20a8-48de-905b-1e32f57ee10a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.891169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa406c8070341dda5edc52d31a25194', 'request_id': 'req-cebd28b9-20a8-48de-905b-1e32f57ee10a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.908826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] The token cache is being invalidate because project 4fa406c8070341dda5edc52d31a25194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:15.909156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.909976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cebd28b9-20a8-48de-905b-1e32f57ee10a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:15.915256 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1038/2072] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:15 2022] DELETE /identity/v3/projects/4fa406c8070341dda5edc52d31a25194 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:15.917200 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4335f-8706-42ee-aaa8-b670ead5e3da None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:15.917526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4335f-8706-42ee-aaa8-b670ead5e3da None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:15.917778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4335f-8706-42ee-aaa8-b670ead5e3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:15.940786 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d9a4335f-8706-42ee-aaa8-b670ead5e3da None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:15.942416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d9a4335f-8706-42ee-aaa8-b670ead5e3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:16.010549 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1035/2073] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:16.018938 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9ed73-6aad-49a2-90c4-c1eaf59bcab5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.019409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9ed73-6aad-49a2-90c4-c1eaf59bcab5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.019696 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9ed73-6aad-49a2-90c4-c1eaf59bcab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.041446 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-eca9ed73-6aad-49a2-90c4-c1eaf59bcab5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:16.042667 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-eca9ed73-6aad-49a2-90c4-c1eaf59bcab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:16.052188 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1039/2074] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:16.056552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4335f-8706-42ee-aaa8-b670ead5e3da None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:16.068387 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4335f-8706-42ee-aaa8-b670ead5e3da None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:16.074019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bec20e-a352-40ca-9439-2d52ae14b7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:16.074740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bec20e-a352-40ca-9439-2d52ae14b7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.075176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bec20e-a352-40ca-9439-2d52ae14b7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.075515 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bec20e-a352-40ca-9439-2d52ae14b7a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.076311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bec20e-a352-40ca-9439-2d52ae14b7a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:55:16.076661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bec20e-a352-40ca-9439-2d52ae14b7a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:16.078438 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bec20e-a352-40ca-9439-2d52ae14b7a7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:16.097567 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1036/2075] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:55:16.098881 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9ed73-6aad-49a2-90c4-c1eaf59bcab5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:16.120635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9ed73-6aad-49a2-90c4-c1eaf59bcab5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:16.125312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b2eb-a37d-4181-85b4-927ef9b473f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:16.126019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b2eb-a37d-4181-85b4-927ef9b473f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.126299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b2eb-a37d-4181-85b4-927ef9b473f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.126566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b2eb-a37d-4181-85b4-927ef9b473f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.127231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149b2eb-a37d-4181-85b4-927ef9b473f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:16.129170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149b2eb-a37d-4181-85b4-927ef9b473f2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:16.161065 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1040/2076] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:16 2022] POST /identity/v3/projects => generated 361 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:16.162508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bec20e-a352-40ca-9439-2d52ae14b7a7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:16.174094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bec20e-a352-40ca-9439-2d52ae14b7a7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:16.239198 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7bb45-5298-43fc-870b-b79bd30e15aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:16.239818 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7bb45-5298-43fc-870b-b79bd30e15aa admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.240092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7bb45-5298-43fc-870b-b79bd30e15aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.240361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7bb45-5298-43fc-870b-b79bd30e15aa admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.241028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7bb45-5298-43fc-870b-b79bd30e15aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:16.242951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7bb45-5298-43fc-870b-b79bd30e15aa admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:16.267596 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1037/2077] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:16 2022] POST /identity/v3/users => generated 281 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:16.271707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b2eb-a37d-4181-85b4-927ef9b473f2 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:16.287456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b2eb-a37d-4181-85b4-927ef9b473f2 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:16.292443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c428d-44e3-4664-b677-c386b3ec3b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:16.293057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c428d-44e3-4664-b677-c386b3ec3b32 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.293553 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c428d-44e3-4664-b677-c386b3ec3b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.293824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c428d-44e3-4664-b677-c386b3ec3b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.294473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c428d-44e3-4664-b677-c386b3ec3b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:16.296144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c428d-44e3-4664-b677-c386b3ec3b32 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:16.307465 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1041/2078] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:16.311619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7bb45-5298-43fc-870b-b79bd30e15aa admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:16.321695 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7bb45-5298-43fc-870b-b79bd30e15aa admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:16.333258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc437c39-a480-41c6-8329-f9d418144718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:16.333877 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc437c39-a480-41c6-8329-f9d418144718 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.334163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc437c39-a480-41c6-8329-f9d418144718 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.334412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc437c39-a480-41c6-8329-f9d418144718 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.337766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc437c39-a480-41c6-8329-f9d418144718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:16.339475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc437c39-a480-41c6-8329-f9d418144718 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:16.354437 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1038/2079] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:55:16.358491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c428d-44e3-4664-b677-c386b3ec3b32 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:16.369185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c428d-44e3-4664-b677-c386b3ec3b32 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:16.373985 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954c65f-b36f-4a8e-9bbf-7dfc7e24f3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:16.374670 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954c65f-b36f-4a8e-9bbf-7dfc7e24f3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.375005 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954c65f-b36f-4a8e-9bbf-7dfc7e24f3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.375279 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954c65f-b36f-4a8e-9bbf-7dfc7e24f3d8 admin admin] PATH_INFO: `/v3/projects/38662ebf9d104fd1a39c6fb14848cf46/users/1b03218a8f8c4837b94b59eeb7def03b/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.391522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e954c65f-b36f-4a8e-9bbf-7dfc7e24f3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38662ebf9d104fd1a39c6fb14848cf46, user_id=1b03218a8f8c4837b94b59eeb7def03b, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:16.394316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e954c65f-b36f-4a8e-9bbf-7dfc7e24f3d8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:16.409693 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1042/2080] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:16 2022] PUT /identity/v3/projects/38662ebf9d104fd1a39c6fb14848cf46/users/1b03218a8f8c4837b94b59eeb7def03b/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:16.421471 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ff1e8-de1e-4f8c-ae8b-cc50ab032366 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.422282 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ff1e8-de1e-4f8c-ae8b-cc50ab032366 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.422662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ff1e8-de1e-4f8c-ae8b-cc50ab032366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.431605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:16.456346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-186ff1e8-de1e-4f8c-ae8b-cc50ab032366 None None] MFA Rules not processed for user `1b03218a8f8c4837b94b59eeb7def03b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:16.457574 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-186ff1e8-de1e-4f8c-ae8b-cc50ab032366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:16.528017 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1039/2081] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:16.563949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954c65f-b36f-4a8e-9bbf-7dfc7e24f3d8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:16.565915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e954c65f-b36f-4a8e-9bbf-7dfc7e24f3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:16.624316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954c65f-b36f-4a8e-9bbf-7dfc7e24f3d8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:16.627987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7788b1c-bb14-4127-9747-ecda94349626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:16.628588 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7788b1c-bb14-4127-9747-ecda94349626 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.628863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7788b1c-bb14-4127-9747-ecda94349626 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.629130 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7788b1c-bb14-4127-9747-ecda94349626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.637827 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7788b1c-bb14-4127-9747-ecda94349626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:16.639297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7788b1c-bb14-4127-9747-ecda94349626 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:16.648674 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1043/2082] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:16.666310 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ff1e8-de1e-4f8c-ae8b-cc50ab032366 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:16.732324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ff1e8-de1e-4f8c-ae8b-cc50ab032366 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:16.737148 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:16.737787 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.738064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.738335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] PATH_INFO: `/v3/users/1b03218a8f8c4837b94b59eeb7def03b` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.739491 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b03218a8f8c4837b94b59eeb7def03b)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:16.741414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:16.790542 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b03218a8f8c4837b94b59eeb7def03b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:16.791262 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b03218a8f8c4837b94b59eeb7def03b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:16.802691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b03218a8f8c4837b94b59eeb7def03b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:16.811311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b03218a8f8c4837b94b59eeb7def03b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:16.824174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b03218a8f8c4837b94b59eeb7def03b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:16.831200 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1040/2083] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:16 2022] DELETE /identity/v3/users/1b03218a8f8c4837b94b59eeb7def03b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:16.835921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f423-07b7-496d-a736-748817ed7c1a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.836199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f423-07b7-496d-a736-748817ed7c1a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.836473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f423-07b7-496d-a736-748817ed7c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:16.856675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6f38f423-07b7-496d-a736-748817ed7c1a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:16.857910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6f38f423-07b7-496d-a736-748817ed7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:16.910249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:16.922751 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1044/2084] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:16.929954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:16.931502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:16.945459 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:16.992303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f87cae-a0bf-4ae9-aeed-cabe2ca040bc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:16.995989 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f8265-14b6-49e0-ab96-f4c5f6e586b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:16.996600 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f8265-14b6-49e0-ab96-f4c5f6e586b4 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:16.996885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f8265-14b6-49e0-ab96-f4c5f6e586b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:16.997159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f8265-14b6-49e0-ab96-f4c5f6e586b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.006058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f8265-14b6-49e0-ab96-f4c5f6e586b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:17.007608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f8265-14b6-49e0-ab96-f4c5f6e586b4 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:17.017351 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1041/2085] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:17.293039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38f423-07b7-496d-a736-748817ed7c1a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:17.294854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6f38f423-07b7-496d-a736-748817ed7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:17.311459 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38f423-07b7-496d-a736-748817ed7c1a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:17.316989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:17.317738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.318016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.319388 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] PATH_INFO: `/v3/projects/38662ebf9d104fd1a39c6fb14848cf46` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.320911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38662ebf9d104fd1a39c6fb14848cf46)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:17.323125 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:17.374471 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38662ebf9d104fd1a39c6fb14848cf46', 'request_id': 'req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:17.380337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38662ebf9d104fd1a39c6fb14848cf46', 'request_id': 'req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:17.386480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38662ebf9d104fd1a39c6fb14848cf46', 'request_id': 'req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:17.399951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] The token cache is being invalidate because project 38662ebf9d104fd1a39c6fb14848cf46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:17.400266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:17.400906 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00db5a59-580a-4d0e-a0b7-4f3ae7fc4915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:17.402393 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1045/2086] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:17 2022] DELETE /identity/v3/projects/38662ebf9d104fd1a39c6fb14848cf46 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:17.407305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4ec4-34b5-40ff-b82e-88eef9628d0f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.407624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4ec4-34b5-40ff-b82e-88eef9628d0f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.407900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4ec4-34b5-40ff-b82e-88eef9628d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.427933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c18e4ec4-34b5-40ff-b82e-88eef9628d0f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:17.429153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c18e4ec4-34b5-40ff-b82e-88eef9628d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:17.492306 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1042/2087] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:17.500189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df716cb-5ab8-42ae-ad77-a6e414b09509 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.500475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df716cb-5ab8-42ae-ad77-a6e414b09509 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.500794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df716cb-5ab8-42ae-ad77-a6e414b09509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.520236 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1df716cb-5ab8-42ae-ad77-a6e414b09509 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:17.521457 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1df716cb-5ab8-42ae-ad77-a6e414b09509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:17.529542 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1046/2088] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:17.533818 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e4ec4-34b5-40ff-b82e-88eef9628d0f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:17.543880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e4ec4-34b5-40ff-b82e-88eef9628d0f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:17.549808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee6009-fb49-4a5c-a71e-295333782d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:17.550400 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee6009-fb49-4a5c-a71e-295333782d73 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.550672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee6009-fb49-4a5c-a71e-295333782d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.550969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee6009-fb49-4a5c-a71e-295333782d73 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.551607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee6009-fb49-4a5c-a71e-295333782d73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:55:17.551909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee6009-fb49-4a5c-a71e-295333782d73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:17.553454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee6009-fb49-4a5c-a71e-295333782d73 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:17.567236 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1043/2089] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:55:17.571671 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df716cb-5ab8-42ae-ad77-a6e414b09509 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:17.581025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df716cb-5ab8-42ae-ad77-a6e414b09509 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:17.586042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f145cb-ed10-41a5-b45d-6b97b845c667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:17.586636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f145cb-ed10-41a5-b45d-6b97b845c667 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.586943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f145cb-ed10-41a5-b45d-6b97b845c667 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.587278 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f145cb-ed10-41a5-b45d-6b97b845c667 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.587922 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f145cb-ed10-41a5-b45d-6b97b845c667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:17.589817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f145cb-ed10-41a5-b45d-6b97b845c667 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:17.619606 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1047/2090] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:17 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:17.623756 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee6009-fb49-4a5c-a71e-295333782d73 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:17.633908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee6009-fb49-4a5c-a71e-295333782d73 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:17.692945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c98a9b-62eb-4a82-970e-77f713102fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:17.693543 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98a9b-62eb-4a82-970e-77f713102fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.693813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98a9b-62eb-4a82-970e-77f713102fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.694077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c98a9b-62eb-4a82-970e-77f713102fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.694741 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c98a9b-62eb-4a82-970e-77f713102fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:17.696705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c98a9b-62eb-4a82-970e-77f713102fe4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:17.720274 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1044/2091] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:17 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:17.724329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f145cb-ed10-41a5-b45d-6b97b845c667 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:17.735424 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f145cb-ed10-41a5-b45d-6b97b845c667 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:17.740695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41429843-73c6-484e-84db-d5c709a436ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:17.741329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41429843-73c6-484e-84db-d5c709a436ad admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.741603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41429843-73c6-484e-84db-d5c709a436ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.741840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41429843-73c6-484e-84db-d5c709a436ad admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.742507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41429843-73c6-484e-84db-d5c709a436ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:17.744135 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41429843-73c6-484e-84db-d5c709a436ad admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:17.755434 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1048/2092] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:17.759639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c98a9b-62eb-4a82-970e-77f713102fe4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:17.772997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c98a9b-62eb-4a82-970e-77f713102fe4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:17.779008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fd56d-7e9a-4bf6-8af5-b4e41fe1bbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:17.779622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fd56d-7e9a-4bf6-8af5-b4e41fe1bbdb admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.779899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fd56d-7e9a-4bf6-8af5-b4e41fe1bbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.780170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fd56d-7e9a-4bf6-8af5-b4e41fe1bbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.780831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8fd56d-7e9a-4bf6-8af5-b4e41fe1bbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:17.782480 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8fd56d-7e9a-4bf6-8af5-b4e41fe1bbdb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:17.794503 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1045/2093] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:17.799154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41429843-73c6-484e-84db-d5c709a436ad admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:17.809241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41429843-73c6-484e-84db-d5c709a436ad admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:17.814065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e028e47-8140-4c0e-989a-cd4b14db75aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:17.814735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e028e47-8140-4c0e-989a-cd4b14db75aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.815025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e028e47-8140-4c0e-989a-cd4b14db75aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.815296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e028e47-8140-4c0e-989a-cd4b14db75aa admin admin] PATH_INFO: `/v3/projects/fdc6c6bf52dd48f3ae2163fe7a183a8e/users/7d5e24245ed64b5f9dad7a1a20b3366b/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.830026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e028e47-8140-4c0e-989a-cd4b14db75aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc6c6bf52dd48f3ae2163fe7a183a8e, user_id=7d5e24245ed64b5f9dad7a1a20b3366b, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:17.832727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e028e47-8140-4c0e-989a-cd4b14db75aa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:17.846498 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1049/2094] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:17 2022] PUT /identity/v3/projects/fdc6c6bf52dd48f3ae2163fe7a183a8e/users/7d5e24245ed64b5f9dad7a1a20b3366b/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:55:17.858418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb3005-ad77-447d-b1d7-c32db7232216 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:17.858655 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb3005-ad77-447d-b1d7-c32db7232216 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:17.859019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb3005-ad77-447d-b1d7-c32db7232216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:17.888295 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c2bb3005-ad77-447d-b1d7-c32db7232216 None None] MFA Rules not processed for user `7d5e24245ed64b5f9dad7a1a20b3366b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:17.889511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c2bb3005-ad77-447d-b1d7-c32db7232216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:17.898907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:17.954726 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1046/2095] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:17.966759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e028e47-8140-4c0e-989a-cd4b14db75aa admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:17.969060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0e028e47-8140-4c0e-989a-cd4b14db75aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:18.027546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e028e47-8140-4c0e-989a-cd4b14db75aa admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:18.031971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2384a-cf67-40fb-9bea-f4c83c7f55a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:18.032585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2384a-cf67-40fb-9bea-f4c83c7f55a2 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:18.032862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2384a-cf67-40fb-9bea-f4c83c7f55a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:18.033134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2384a-cf67-40fb-9bea-f4c83c7f55a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:18.042063 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2384a-cf67-40fb-9bea-f4c83c7f55a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:18.043531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2384a-cf67-40fb-9bea-f4c83c7f55a2 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:18.052934 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1050/2096] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:23.754096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1579f07-d1fb-433f-9f8b-9d7b507eff7f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:23.754615 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1579f07-d1fb-433f-9f8b-9d7b507eff7f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:23.754711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1579f07-d1fb-433f-9f8b-9d7b507eff7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:23.775236 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e1579f07-d1fb-433f-9f8b-9d7b507eff7f None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:23.776739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e1579f07-d1fb-433f-9f8b-9d7b507eff7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:23.834122 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1047/2097] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:55:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:23.846531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2384a-cf67-40fb-9bea-f4c83c7f55a2 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:23.848055 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-74a2384a-cf67-40fb-9bea-f4c83c7f55a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:23.874219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:23.920284 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2384a-cf67-40fb-9bea-f4c83c7f55a2 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:23.925644 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c796-3004-464f-9725-396167f18c28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:23.926253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c796-3004-464f-9725-396167f18c28 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:23.926529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c796-3004-464f-9725-396167f18c28 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:23.926831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c796-3004-464f-9725-396167f18c28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:23.934994 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c796-3004-464f-9725-396167f18c28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:23.936945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c796-3004-464f-9725-396167f18c28 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:23.947206 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1051/2098] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:55:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:55:27.722967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1579f07-d1fb-433f-9f8b-9d7b507eff7f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:27.724481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e1579f07-d1fb-433f-9f8b-9d7b507eff7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:27.780319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1579f07-d1fb-433f-9f8b-9d7b507eff7f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:27.784845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c43357-9fd7-42ba-a1b6-e83231876c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:27.785508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43357-9fd7-42ba-a1b6-e83231876c5d service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:27.785792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43357-9fd7-42ba-a1b6-e83231876c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:27.786107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43357-9fd7-42ba-a1b6-e83231876c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:27.787992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d6c43357-9fd7-42ba-a1b6-e83231876c5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:27.857147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c43357-9fd7-42ba-a1b6-e83231876c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:27.858705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c43357-9fd7-42ba-a1b6-e83231876c5d service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:27.874180 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1048/2099] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:32.237958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c796-3004-464f-9725-396167f18c28 service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:32.237958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-13d1c796-3004-464f-9725-396167f18c28 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:32.392104 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c43357-9fd7-42ba-a1b6-e83231876c5d service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:32.392778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c796-3004-464f-9725-396167f18c28 service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:32.400689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:32.402343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:32.403125 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:32.403453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:32.403831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] PATH_INFO: `/v3/users/7d793a1f26a3467ea8119e27e3394371` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:32.404944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d793a1f26a3467ea8119e27e3394371)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:32.406913 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:32.471013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c43357-9fd7-42ba-a1b6-e83231876c5d service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:32.478265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:32.479544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:32.481217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:32.481759 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] PATH_INFO: `/v3/users/7d5e24245ed64b5f9dad7a1a20b3366b` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:32.484009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d5e24245ed64b5f9dad7a1a20b3366b)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:32.486749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:32.503320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d793a1f26a3467ea8119e27e3394371'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.524114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d793a1f26a3467ea8119e27e3394371'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.525157 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d793a1f26a3467ea8119e27e3394371'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.535546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d793a1f26a3467ea8119e27e3394371'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.553449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78bf04fc-979f-4567-9bdb-2d680db6dc4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d793a1f26a3467ea8119e27e3394371'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.565594 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1052/2100] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:32 2022] DELETE /identity/v3/users/7d793a1f26a3467ea8119e27e3394371 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:32.570999 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d5e24245ed64b5f9dad7a1a20b3366b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.572103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d5e24245ed64b5f9dad7a1a20b3366b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.580899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d5e24245ed64b5f9dad7a1a20b3366b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.609036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d5e24245ed64b5f9dad7a1a20b3366b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.629253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d5e24245ed64b5f9dad7a1a20b3366b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:32.638116 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1049/2101] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:32 2022] DELETE /identity/v3/users/7d5e24245ed64b5f9dad7a1a20b3366b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:32.642524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174c709-d433-4d06-b80c-f53416efcd44 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:32.643137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174c709-d433-4d06-b80c-f53416efcd44 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:32.643636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174c709-d433-4d06-b80c-f53416efcd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:32.742005 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b174c709-d433-4d06-b80c-f53416efcd44 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:32.744772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b174c709-d433-4d06-b80c-f53416efcd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:32.788293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:32.912423 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1053/2102] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:32.931555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:32.931867 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:32.957587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174c709-d433-4d06-b80c-f53416efcd44 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:32.959269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b174c709-d433-4d06-b80c-f53416efcd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:32.990025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174c709-d433-4d06-b80c-f53416efcd44 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.008335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.009328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.011716 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.012201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] PATH_INFO: `/v3/projects/e3b77fd80fd84c29b4581fda1ff8cef9` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.014173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b77fd80fd84c29b4581fda1ff8cef9)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.016331 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.084619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b77fd80fd84c29b4581fda1ff8cef9', 'request_id': 'req-b26693fb-469b-4f37-857a-09b044c09c60'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.086966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d80e13-24fe-4737-9b89-d53fdb652f25 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.090732 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b77fd80fd84c29b4581fda1ff8cef9', 'request_id': 'req-b26693fb-469b-4f37-857a-09b044c09c60'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.098489 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b77fd80fd84c29b4581fda1ff8cef9', 'request_id': 'req-b26693fb-469b-4f37-857a-09b044c09c60'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.117511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] The token cache is being invalidate because project e3b77fd80fd84c29b4581fda1ff8cef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:33.117867 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.118584 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b26693fb-469b-4f37-857a-09b044c09c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.120179 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1054/2103] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:32 2022] DELETE /identity/v3/projects/e3b77fd80fd84c29b4581fda1ff8cef9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:55:33.125721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325f632-3e59-48f5-9931-5dfdd949b078 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.126066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325f632-3e59-48f5-9931-5dfdd949b078 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.126509 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325f632-3e59-48f5-9931-5dfdd949b078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.134035 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:33.154469 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53b450-f848-4611-909b-617658718849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.155228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53b450-f848-4611-909b-617658718849 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.155709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53b450-f848-4611-909b-617658718849 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.155919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53b450-f848-4611-909b-617658718849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.157614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6b53b450-f848-4611-909b-617658718849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:33.179253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b325f632-3e59-48f5-9931-5dfdd949b078 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:33.180435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b325f632-3e59-48f5-9931-5dfdd949b078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:33.225417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53b450-f848-4611-909b-617658718849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.227398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53b450-f848-4611-909b-617658718849 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.237336 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1050/2104] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:33.239303 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1055/2105] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:55:33.247159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da12151-326e-4cd6-9435-fb2e4946be94 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.247644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da12151-326e-4cd6-9435-fb2e4946be94 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.247920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da12151-326e-4cd6-9435-fb2e4946be94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.275203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3da12151-326e-4cd6-9435-fb2e4946be94 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:33.276500 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3da12151-326e-4cd6-9435-fb2e4946be94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:33.286407 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1051/2106] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:33.291021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325f632-3e59-48f5-9931-5dfdd949b078 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.302286 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325f632-3e59-48f5-9931-5dfdd949b078 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.307633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d3108-2cf8-4140-bf3d-f290982ac013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.308278 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d3108-2cf8-4140-bf3d-f290982ac013 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.308684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d3108-2cf8-4140-bf3d-f290982ac013 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.309059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d3108-2cf8-4140-bf3d-f290982ac013 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.310063 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d3108-2cf8-4140-bf3d-f290982ac013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:55:33.310408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d3108-2cf8-4140-bf3d-f290982ac013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.312285 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d3108-2cf8-4140-bf3d-f290982ac013 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.328137 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1056/2107] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:55:33.329255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da12151-326e-4cd6-9435-fb2e4946be94 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.338850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da12151-326e-4cd6-9435-fb2e4946be94 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.344127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2d68e-6b52-4f79-9fd9-8532794b9088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.344701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2d68e-6b52-4f79-9fd9-8532794b9088 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.345000 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2d68e-6b52-4f79-9fd9-8532794b9088 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.345343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2d68e-6b52-4f79-9fd9-8532794b9088 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.345984 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2d68e-6b52-4f79-9fd9-8532794b9088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.347896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2d68e-6b52-4f79-9fd9-8532794b9088 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.374624 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1052/2108] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:33 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:33.379368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d3108-2cf8-4140-bf3d-f290982ac013 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.384961 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:33.395323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d3108-2cf8-4140-bf3d-f290982ac013 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.455818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94f6f0-443f-4bc4-8371-8711c641d50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.456443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94f6f0-443f-4bc4-8371-8711c641d50f admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.456749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94f6f0-443f-4bc4-8371-8711c641d50f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.457006 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94f6f0-443f-4bc4-8371-8711c641d50f admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.457645 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94f6f0-443f-4bc4-8371-8711c641d50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.459601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94f6f0-443f-4bc4-8371-8711c641d50f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.486887 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1057/2109] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:33 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:33.488573 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2d68e-6b52-4f79-9fd9-8532794b9088 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.500436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2d68e-6b52-4f79-9fd9-8532794b9088 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.505847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbb40d-6f74-42da-afdb-85b6686d37e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.506476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbb40d-6f74-42da-afdb-85b6686d37e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.506810 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbb40d-6f74-42da-afdb-85b6686d37e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.507134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbb40d-6f74-42da-afdb-85b6686d37e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.507739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbb40d-6f74-42da-afdb-85b6686d37e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.509488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbb40d-6f74-42da-afdb-85b6686d37e6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.521407 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1053/2110] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:33.522507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94f6f0-443f-4bc4-8371-8711c641d50f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.524694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2c94f6f0-443f-4bc4-8371-8711c641d50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:33.525333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbb40d-6f74-42da-afdb-85b6686d37e6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.536245 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbb40d-6f74-42da-afdb-85b6686d37e6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.540215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94f6f0-443f-4bc4-8371-8711c641d50f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.541150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4b8eb-a1ea-4321-8bd8-8b2fc91829ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.545765 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.546486 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.547707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.547707 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] PATH_INFO: `/v3/projects/fdc6c6bf52dd48f3ae2163fe7a183a8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.547945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4b8eb-a1ea-4321-8bd8-8b2fc91829ab admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.547945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4b8eb-a1ea-4321-8bd8-8b2fc91829ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.548287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4b8eb-a1ea-4321-8bd8-8b2fc91829ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.548890 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc6c6bf52dd48f3ae2163fe7a183a8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.549001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4b8eb-a1ea-4321-8bd8-8b2fc91829ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.550860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4b8eb-a1ea-4321-8bd8-8b2fc91829ab admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.550987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.562818 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1054/2111] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:33.568078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4b8eb-a1ea-4321-8bd8-8b2fc91829ab admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.578949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4b8eb-a1ea-4321-8bd8-8b2fc91829ab admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.584544 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5b4be-5d69-4fd9-953b-e08382ddb536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.585333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5b4be-5d69-4fd9-953b-e08382ddb536 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.585713 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5b4be-5d69-4fd9-953b-e08382ddb536 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.586074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5b4be-5d69-4fd9-953b-e08382ddb536 admin admin] PATH_INFO: `/v3/projects/ff59dc49e8a945d3936f85d58c49774e/users/8b7cf0929a9c433ebde5a292a8f0f518/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.601607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5b4be-5d69-4fd9-953b-e08382ddb536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff59dc49e8a945d3936f85d58c49774e, user_id=8b7cf0929a9c433ebde5a292a8f0f518, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.602915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc6c6bf52dd48f3ae2163fe7a183a8e', 'request_id': 'req-d1417354-06db-4604-a3c1-5156535036ba'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.604301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5b4be-5d69-4fd9-953b-e08382ddb536 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.608582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc6c6bf52dd48f3ae2163fe7a183a8e', 'request_id': 'req-d1417354-06db-4604-a3c1-5156535036ba'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.616146 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc6c6bf52dd48f3ae2163fe7a183a8e', 'request_id': 'req-d1417354-06db-4604-a3c1-5156535036ba'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.618844 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1055/2112] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:33 2022] PUT /identity/v3/projects/ff59dc49e8a945d3936f85d58c49774e/users/8b7cf0929a9c433ebde5a292a8f0f518/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:55:33.629617 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] The token cache is being invalidate because project fdc6c6bf52dd48f3ae2163fe7a183a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:33.629911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.630386 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221f447-3fe4-4031-bcc2-363a73044577 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.630564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d1417354-06db-4604-a3c1-5156535036ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:33.630674 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221f447-3fe4-4031-bcc2-363a73044577 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.630958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221f447-3fe4-4031-bcc2-363a73044577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.632170 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1058/2113] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:33 2022] DELETE /identity/v3/projects/fdc6c6bf52dd48f3ae2163fe7a183a8e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:33.636847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c37342-70a6-4b78-af5f-77f4881ef219 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.637081 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c37342-70a6-4b78-af5f-77f4881ef219 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.637360 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c37342-70a6-4b78-af5f-77f4881ef219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.658130 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-08c37342-70a6-4b78-af5f-77f4881ef219 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:33.659363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-08c37342-70a6-4b78-af5f-77f4881ef219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:33.660652 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1221f447-3fe4-4031-bcc2-363a73044577 None None] MFA Rules not processed for user `8b7cf0929a9c433ebde5a292a8f0f518`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:33.661829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1221f447-3fe4-4031-bcc2-363a73044577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:33.721330 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1059/2114] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:55:33.728891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636f2b8-75d1-4e68-9d22-69c76a69723b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.729184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636f2b8-75d1-4e68-9d22-69c76a69723b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.729459 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636f2b8-75d1-4e68-9d22-69c76a69723b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.729987 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1056/2115] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:33.741944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221f447-3fe4-4031-bcc2-363a73044577 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.743329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1221f447-3fe4-4031-bcc2-363a73044577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:33.748944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:33.749672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c636f2b8-75d1-4e68-9d22-69c76a69723b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:33.750897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c636f2b8-75d1-4e68-9d22-69c76a69723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:33.763106 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1060/2116] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:55:33.764087 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636f2b8-75d1-4e68-9d22-69c76a69723b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.773137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636f2b8-75d1-4e68-9d22-69c76a69723b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.777754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6c497-e41b-418c-968d-35506ad9049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.778345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6c497-e41b-418c-968d-35506ad9049e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.778686 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6c497-e41b-418c-968d-35506ad9049e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.778994 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6c497-e41b-418c-968d-35506ad9049e admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.779631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6c497-e41b-418c-968d-35506ad9049e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:55:33.779932 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6c497-e41b-418c-968d-35506ad9049e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.781536 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6c497-e41b-418c-968d-35506ad9049e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.783860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:33.795411 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1061/2117] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:55:33.799960 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6c497-e41b-418c-968d-35506ad9049e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.810479 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6c497-e41b-418c-968d-35506ad9049e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.815084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35631639-9552-4cbf-950a-5b2e379f9311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.815668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35631639-9552-4cbf-950a-5b2e379f9311 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.815949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35631639-9552-4cbf-950a-5b2e379f9311 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.816250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35631639-9552-4cbf-950a-5b2e379f9311 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.816876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35631639-9552-4cbf-950a-5b2e379f9311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.818786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35631639-9552-4cbf-950a-5b2e379f9311 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.847153 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1062/2118] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:33 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:33.850217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221f447-3fe4-4031-bcc2-363a73044577 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.851445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35631639-9552-4cbf-950a-5b2e379f9311 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.914835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35631639-9552-4cbf-950a-5b2e379f9311 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.915049 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6601-6913-4da6-99b3-bae82ec39e8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.915637 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6601-6913-4da6-99b3-bae82ec39e8e service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.915938 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6601-6913-4da6-99b3-bae82ec39e8e service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.916192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6601-6913-4da6-99b3-bae82ec39e8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.919932 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e6ade-0b06-4993-b7d8-65a8ae8f537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.920530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6ade-0b06-4993-b7d8-65a8ae8f537b admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.920805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6ade-0b06-4993-b7d8-65a8ae8f537b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.921419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6ade-0b06-4993-b7d8-65a8ae8f537b admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.922050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e6ade-0b06-4993-b7d8-65a8ae8f537b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.924302 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e6ade-0b06-4993-b7d8-65a8ae8f537b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.946761 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1063/2119] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:33 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:33.950833 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e6ade-0b06-4993-b7d8-65a8ae8f537b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.960274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e6ade-0b06-4993-b7d8-65a8ae8f537b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.964842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8a41d-2903-42d6-befd-3174d92759a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.965418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8a41d-2903-42d6-befd-3174d92759a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.965694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8a41d-2903-42d6-befd-3174d92759a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.965963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8a41d-2903-42d6-befd-3174d92759a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.966967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8a41d-2903-42d6-befd-3174d92759a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.968577 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8a41d-2903-42d6-befd-3174d92759a5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.974619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca6601-6913-4da6-99b3-bae82ec39e8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:33.976279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca6601-6913-4da6-99b3-bae82ec39e8e service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:33.978492 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1064/2120] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:33.982398 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8a41d-2903-42d6-befd-3174d92759a5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:33.989127 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1057/2121] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:55:33.992658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8a41d-2903-42d6-befd-3174d92759a5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:33.997754 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ab5fe-c612-4d09-a5b3-559d8ee1ae2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:33.998352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab5fe-c612-4d09-a5b3-559d8ee1ae2a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:33.998635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab5fe-c612-4d09-a5b3-559d8ee1ae2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:33.998949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab5fe-c612-4d09-a5b3-559d8ee1ae2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:33.999720 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ab5fe-c612-4d09-a5b3-559d8ee1ae2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:34.001305 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ab5fe-c612-4d09-a5b3-559d8ee1ae2a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:34.020799 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1065/2122] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:55:34.024234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6601-6913-4da6-99b3-bae82ec39e8e service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:34.040527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6601-6913-4da6-99b3-bae82ec39e8e service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:34.051589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17e924-cb0f-4b77-8150-3dd2824845c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:34.051589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17e924-cb0f-4b77-8150-3dd2824845c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:34.051589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17e924-cb0f-4b77-8150-3dd2824845c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:34.051589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17e924-cb0f-4b77-8150-3dd2824845c8 admin admin] PATH_INFO: `/v3/projects/48ae99be2f1944bb89c5291ecb6ee02a/users/31d493ed96b74a4e8d897dcea947ac0e/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:34.065303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:34.071390 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17e924-cb0f-4b77-8150-3dd2824845c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ae99be2f1944bb89c5291ecb6ee02a, user_id=31d493ed96b74a4e8d897dcea947ac0e, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:34.074010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17e924-cb0f-4b77-8150-3dd2824845c8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:34.090604 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1058/2123] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:34 2022] PUT /identity/v3/projects/48ae99be2f1944bb89c5291ecb6ee02a/users/31d493ed96b74a4e8d897dcea947ac0e/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:34.104083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f268e-36cb-466c-a827-363129d81a9e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:34.104558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f268e-36cb-466c-a827-363129d81a9e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:34.104995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f268e-36cb-466c-a827-363129d81a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:34.133770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-750f268e-36cb-466c-a827-363129d81a9e None None] MFA Rules not processed for user `31d493ed96b74a4e8d897dcea947ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:34.135361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-750f268e-36cb-466c-a827-363129d81a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:34.219668 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1066/2124] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:34.222889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17e924-cb0f-4b77-8150-3dd2824845c8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:34.224990 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1e17e924-cb0f-4b77-8150-3dd2824845c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:34.231376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f268e-36cb-466c-a827-363129d81a9e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:34.232965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-750f268e-36cb-466c-a827-363129d81a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:34.271120 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:34.291054 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17e924-cb0f-4b77-8150-3dd2824845c8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:34.295598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f268e-36cb-466c-a827-363129d81a9e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:34.295900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:34.296531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:34.296811 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:34.297087 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] PATH_INFO: `/v3/users/8b7cf0929a9c433ebde5a292a8f0f518` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:34.298533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7cf0929a9c433ebde5a292a8f0f518)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:34.299590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197f075-07bb-4412-9992-ebceab307bb8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b1c2ea68e4741728469ab7cc542b6d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:34.300217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197f075-07bb-4412-9992-ebceab307bb8 service glance] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:34.300473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197f075-07bb-4412-9992-ebceab307bb8 service glance] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:34.300581 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:34.300746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197f075-07bb-4412-9992-ebceab307bb8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:34.309085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197f075-07bb-4412-9992-ebceab307bb8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:34.310530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197f075-07bb-4412-9992-ebceab307bb8 service glance] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:34.319310 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1067/2125] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:55:34.351903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7cf0929a9c433ebde5a292a8f0f518'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.352553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7cf0929a9c433ebde5a292a8f0f518'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.359870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7cf0929a9c433ebde5a292a8f0f518'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.368621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7cf0929a9c433ebde5a292a8f0f518'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.382506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7cf0929a9c433ebde5a292a8f0f518'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.389646 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1059/2126] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:34 2022] DELETE /identity/v3/users/8b7cf0929a9c433ebde5a292a8f0f518 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:34.394089 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea0665-00ba-4d2d-a602-72a26fc24e95 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:34.394386 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea0665-00ba-4d2d-a602-72a26fc24e95 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:34.394665 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea0665-00ba-4d2d-a602-72a26fc24e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:34.414582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-04ea0665-00ba-4d2d-a602-72a26fc24e95 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:34.415885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-04ea0665-00ba-4d2d-a602-72a26fc24e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:34.476105 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1068/2127] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:34.487057 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:34.488608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:34.548198 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041fa7f-c60c-411f-8887-87643f24b74a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:34.551947 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ccbd8-9dac-4449-ae5a-cd20cdb10cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:34.552548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ccbd8-9dac-4449-ae5a-cd20cdb10cce service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:34.552831 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ccbd8-9dac-4449-ae5a-cd20cdb10cce service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:34.553104 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ccbd8-9dac-4449-ae5a-cd20cdb10cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:34.562299 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ccbd8-9dac-4449-ae5a-cd20cdb10cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:34.563753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ccbd8-9dac-4449-ae5a-cd20cdb10cce service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:34.573083 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1060/2128] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:34.874545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea0665-00ba-4d2d-a602-72a26fc24e95 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:34.876216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-04ea0665-00ba-4d2d-a602-72a26fc24e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:34.890540 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea0665-00ba-4d2d-a602-72a26fc24e95 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:34.895673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:34.896562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:34.896963 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:34.897359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] PATH_INFO: `/v3/projects/ff59dc49e8a945d3936f85d58c49774e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:34.899364 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff59dc49e8a945d3936f85d58c49774e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:34.901626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:34.950706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff59dc49e8a945d3936f85d58c49774e', 'request_id': 'req-78606825-24ab-4c4c-8152-74c6edbebb4a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.956192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff59dc49e8a945d3936f85d58c49774e', 'request_id': 'req-78606825-24ab-4c4c-8152-74c6edbebb4a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.962629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff59dc49e8a945d3936f85d58c49774e', 'request_id': 'req-78606825-24ab-4c4c-8152-74c6edbebb4a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.977447 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] The token cache is being invalidate because project ff59dc49e8a945d3936f85d58c49774e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:34.977568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.978320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78606825-24ab-4c4c-8152-74c6edbebb4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:34.979912 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1069/2129] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:34 2022] DELETE /identity/v3/projects/ff59dc49e8a945d3936f85d58c49774e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:34.985582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff621c91-5b9b-4f3a-a4ed-9b8c6864fa0b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:34.985956 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff621c91-5b9b-4f3a-a4ed-9b8c6864fa0b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:34.986321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff621c91-5b9b-4f3a-a4ed-9b8c6864fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.006846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ff621c91-5b9b-4f3a-a4ed-9b8c6864fa0b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:35.008038 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ff621c91-5b9b-4f3a-a4ed-9b8c6864fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:35.014090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:35.071072 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1061/2130] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:55:35.079123 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a087-443a-4e38-803c-4a99938b859b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.079411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a087-443a-4e38-803c-4a99938b859b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.079684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a087-443a-4e38-803c-4a99938b859b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.100430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b3c6a087-443a-4e38-803c-4a99938b859b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:35.101621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b3c6a087-443a-4e38-803c-4a99938b859b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:35.110544 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1070/2131] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:35.114828 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff621c91-5b9b-4f3a-a4ed-9b8c6864fa0b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.124763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff621c91-5b9b-4f3a-a4ed-9b8c6864fa0b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.130432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc9f79-19e0-4b5f-85ec-f8dfa15d05ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.131024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc9f79-19e0-4b5f-85ec-f8dfa15d05ed admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.131308 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc9f79-19e0-4b5f-85ec-f8dfa15d05ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.131591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc9f79-19e0-4b5f-85ec-f8dfa15d05ed admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.132177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc9f79-19e0-4b5f-85ec-f8dfa15d05ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:55:35.132477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc9f79-19e0-4b5f-85ec-f8dfa15d05ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.134357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc9f79-19e0-4b5f-85ec-f8dfa15d05ed admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.147140 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1062/2132] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:55:35.151721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6a087-443a-4e38-803c-4a99938b859b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.161871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6a087-443a-4e38-803c-4a99938b859b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.168535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c26ecb-a1aa-4fcc-ad48-af33d36ac011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.169343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c26ecb-a1aa-4fcc-ad48-af33d36ac011 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.169679 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c26ecb-a1aa-4fcc-ad48-af33d36ac011 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.169995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c26ecb-a1aa-4fcc-ad48-af33d36ac011 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.170728 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c26ecb-a1aa-4fcc-ad48-af33d36ac011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.173174 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c26ecb-a1aa-4fcc-ad48-af33d36ac011 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.205521 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1071/2133] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:35 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:35.210256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc9f79-19e0-4b5f-85ec-f8dfa15d05ed admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.221396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc9f79-19e0-4b5f-85ec-f8dfa15d05ed admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.297618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b50cf-b086-4ead-8265-83cf291f5506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.298227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b50cf-b086-4ead-8265-83cf291f5506 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.298505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b50cf-b086-4ead-8265-83cf291f5506 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.298876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b50cf-b086-4ead-8265-83cf291f5506 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.299531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b50cf-b086-4ead-8265-83cf291f5506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.301478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b50cf-b086-4ead-8265-83cf291f5506 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.328148 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1063/2134] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:35 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:35.332324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c26ecb-a1aa-4fcc-ad48-af33d36ac011 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.343008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c26ecb-a1aa-4fcc-ad48-af33d36ac011 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.348050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e0d11-4355-4c57-b69f-cc05d6e6899b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.348650 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e0d11-4355-4c57-b69f-cc05d6e6899b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.348927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e0d11-4355-4c57-b69f-cc05d6e6899b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.349536 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e0d11-4355-4c57-b69f-cc05d6e6899b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.350184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e0d11-4355-4c57-b69f-cc05d6e6899b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.352208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e0d11-4355-4c57-b69f-cc05d6e6899b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.364412 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1072/2135] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:35.368409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b50cf-b086-4ead-8265-83cf291f5506 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.379406 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b50cf-b086-4ead-8265-83cf291f5506 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.384133 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d25a73-be7c-4625-a1d1-603bd232d8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.384733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d25a73-be7c-4625-a1d1-603bd232d8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.385009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d25a73-be7c-4625-a1d1-603bd232d8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.385278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d25a73-be7c-4625-a1d1-603bd232d8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.385962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d25a73-be7c-4625-a1d1-603bd232d8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.387583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d25a73-be7c-4625-a1d1-603bd232d8a4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.398269 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1064/2136] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:35.402491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e0d11-4355-4c57-b69f-cc05d6e6899b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.414301 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e0d11-4355-4c57-b69f-cc05d6e6899b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.419507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e201-2a90-466d-8646-0517344bc7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.420176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e201-2a90-466d-8646-0517344bc7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.420448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e201-2a90-466d-8646-0517344bc7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.420712 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e201-2a90-466d-8646-0517344bc7ab admin admin] PATH_INFO: `/v3/projects/e861409abbf4431da49394f68330f5c0/users/6e8e49f0f38d40a3a63775a1edde0bfe/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.437053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3e201-2a90-466d-8646-0517344bc7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e861409abbf4431da49394f68330f5c0, user_id=6e8e49f0f38d40a3a63775a1edde0bfe, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.439955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3e201-2a90-466d-8646-0517344bc7ab admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.442551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:35.456619 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1073/2137] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:35 2022] PUT /identity/v3/projects/e861409abbf4431da49394f68330f5c0/users/6e8e49f0f38d40a3a63775a1edde0bfe/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:55:35.467517 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d4502-e0e9-497e-ad2e-0b8b2b6a066c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.467802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d4502-e0e9-497e-ad2e-0b8b2b6a066c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.468078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d4502-e0e9-497e-ad2e-0b8b2b6a066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.499992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a37d4502-e0e9-497e-ad2e-0b8b2b6a066c None None] MFA Rules not processed for user `6e8e49f0f38d40a3a63775a1edde0bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:35.501218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a37d4502-e0e9-497e-ad2e-0b8b2b6a066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:35.571566 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1065/2138] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:35.576618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e201-2a90-466d-8646-0517344bc7ab admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.649007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e201-2a90-466d-8646-0517344bc7ab admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.654873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9138e-149b-4a1f-a285-5580f73a7235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.655461 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9138e-149b-4a1f-a285-5580f73a7235 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.655744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9138e-149b-4a1f-a285-5580f73a7235 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.656006 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9138e-149b-4a1f-a285-5580f73a7235 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.656647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9138e-149b-4a1f-a285-5580f73a7235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.658679 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9138e-149b-4a1f-a285-5580f73a7235 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.694040 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1074/2139] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:35 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:35.698229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d4502-e0e9-497e-ad2e-0b8b2b6a066c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.709263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d4502-e0e9-497e-ad2e-0b8b2b6a066c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.713027 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:35.770375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7c87b-1ff9-4a74-940e-b5e4e18778a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.771010 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7c87b-1ff9-4a74-940e-b5e4e18778a2 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.771287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7c87b-1ff9-4a74-940e-b5e4e18778a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.771557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7c87b-1ff9-4a74-940e-b5e4e18778a2 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.772184 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7c87b-1ff9-4a74-940e-b5e4e18778a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.774103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7c87b-1ff9-4a74-940e-b5e4e18778a2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.796301 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1066/2140] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:35 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:35.800628 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9138e-149b-4a1f-a285-5580f73a7235 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.810857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9138e-149b-4a1f-a285-5580f73a7235 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.815744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969bf39-8b71-4aae-94a0-45e78cd066fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.816489 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969bf39-8b71-4aae-94a0-45e78cd066fb admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.816863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969bf39-8b71-4aae-94a0-45e78cd066fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.817265 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969bf39-8b71-4aae-94a0-45e78cd066fb admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.818038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969bf39-8b71-4aae-94a0-45e78cd066fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.819685 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969bf39-8b71-4aae-94a0-45e78cd066fb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.833665 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1075/2141] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:35.835350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7c87b-1ff9-4a74-940e-b5e4e18778a2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.845634 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7c87b-1ff9-4a74-940e-b5e4e18778a2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.850210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cf6bd-c025-45aa-b1be-37b9dda58b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.850888 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cf6bd-c025-45aa-b1be-37b9dda58b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.851173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cf6bd-c025-45aa-b1be-37b9dda58b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.851448 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cf6bd-c025-45aa-b1be-37b9dda58b39 admin admin] PATH_INFO: `/v3/projects/71952745078747da8719384c8faf5ecd/users/4384f763c6144904a9fd7d71bad28b2f/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.869011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cf6bd-c025-45aa-b1be-37b9dda58b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71952745078747da8719384c8faf5ecd, user_id=4384f763c6144904a9fd7d71bad28b2f, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.871701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cf6bd-c025-45aa-b1be-37b9dda58b39 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.887823 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1067/2142] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:35 2022] PUT /identity/v3/projects/71952745078747da8719384c8faf5ecd/users/4384f763c6144904a9fd7d71bad28b2f/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:55:35.892014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969bf39-8b71-4aae-94a0-45e78cd066fb admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.902282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969bf39-8b71-4aae-94a0-45e78cd066fb admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.962435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8855c-82f1-4019-b37c-f38ffd99f2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.963000 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8855c-82f1-4019-b37c-f38ffd99f2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.963280 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8855c-82f1-4019-b37c-f38ffd99f2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.963551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8855c-82f1-4019-b37c-f38ffd99f2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.964186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8855c-82f1-4019-b37c-f38ffd99f2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:35.965785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8855c-82f1-4019-b37c-f38ffd99f2a2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:35.975703 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1076/2143] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:35.980022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cf6bd-c025-45aa-b1be-37b9dda58b39 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:35.990876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cf6bd-c025-45aa-b1be-37b9dda58b39 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:35.996280 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2c11b-82e6-4255-b43d-67076f6dc71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:35.996940 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2c11b-82e6-4255-b43d-67076f6dc71f admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:35.997198 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2c11b-82e6-4255-b43d-67076f6dc71f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:35.997553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2c11b-82e6-4255-b43d-67076f6dc71f admin admin] PATH_INFO: `/v3/projects/71952745078747da8719384c8faf5ecd/users/4384f763c6144904a9fd7d71bad28b2f/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:35.999743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2c11b-82e6-4255-b43d-67076f6dc71f admin admin] RBAC: Authorizing `identity:create_grant(project_id=71952745078747da8719384c8faf5ecd, user_id=4384f763c6144904a9fd7d71bad28b2f, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:36.002352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2c11b-82e6-4255-b43d-67076f6dc71f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:36.013305 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1068/2144] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:35 2022] PUT /identity/v3/projects/71952745078747da8719384c8faf5ecd/users/4384f763c6144904a9fd7d71bad28b2f/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:55:36.025437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0505334-295d-4668-8be0-c5b2fdadddbc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:36.025709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0505334-295d-4668-8be0-c5b2fdadddbc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:36.025995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0505334-295d-4668-8be0-c5b2fdadddbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:36.056231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a0505334-295d-4668-8be0-c5b2fdadddbc None None] MFA Rules not processed for user `4384f763c6144904a9fd7d71bad28b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:36.057418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a0505334-295d-4668-8be0-c5b2fdadddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:36.121832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:36.139985 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1077/2145] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:36.152483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2c11b-82e6-4255-b43d-67076f6dc71f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:36.154342 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5fd2c11b-82e6-4255-b43d-67076f6dc71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:36.214384 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2c11b-82e6-4255-b43d-67076f6dc71f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:36.219549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a6830-a667-408b-acb9-70eb2d08eade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:36.220346 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6830-a667-408b-acb9-70eb2d08eade service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:36.220705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6830-a667-408b-acb9-70eb2d08eade service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:36.221059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6830-a667-408b-acb9-70eb2d08eade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:36.234105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a6830-a667-408b-acb9-70eb2d08eade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:36.236294 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a6830-a667-408b-acb9-70eb2d08eade service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:36.250506 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1069/2146] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:36.691889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0505334-295d-4668-8be0-c5b2fdadddbc None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:36.759131 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0505334-295d-4668-8be0-c5b2fdadddbc None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:36.764183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:36.764829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:36.765104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:36.765372 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] PATH_INFO: `/v3/users/6e8e49f0f38d40a3a63775a1edde0bfe` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:36.766538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8e49f0f38d40a3a63775a1edde0bfe)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:36.768492 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:36.816648 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8e49f0f38d40a3a63775a1edde0bfe'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:36.823495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8e49f0f38d40a3a63775a1edde0bfe'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:36.824140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8e49f0f38d40a3a63775a1edde0bfe'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:36.832958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8e49f0f38d40a3a63775a1edde0bfe'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:36.849174 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8e49f0f38d40a3a63775a1edde0bfe'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:36.855952 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1078/2147] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:36 2022] DELETE /identity/v3/users/6e8e49f0f38d40a3a63775a1edde0bfe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:36.860531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151df3d-28a7-4ac1-b96c-9dc56553d0da None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:36.861048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151df3d-28a7-4ac1-b96c-9dc56553d0da None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:36.861048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151df3d-28a7-4ac1-b96c-9dc56553d0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:36.880197 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1151df3d-28a7-4ac1-b96c-9dc56553d0da None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:36.881352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1151df3d-28a7-4ac1-b96c-9dc56553d0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:36.945936 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1070/2148] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:36.955995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:36.957585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:37.017889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbe3d3-e44d-4d7d-9873-53dd5a30a2c8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:37.021695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a756903-c553-4bcd-8816-e68664fb84ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:37.022325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a756903-c553-4bcd-8816-e68664fb84ea service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:37.022613 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a756903-c553-4bcd-8816-e68664fb84ea service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:37.022988 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a756903-c553-4bcd-8816-e68664fb84ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:37.033449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a756903-c553-4bcd-8816-e68664fb84ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:37.034953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a756903-c553-4bcd-8816-e68664fb84ea service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:37.044861 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1079/2149] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:37.317407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151df3d-28a7-4ac1-b96c-9dc56553d0da None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:37.319264 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1151df3d-28a7-4ac1-b96c-9dc56553d0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:37.326602 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:37.336804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151df3d-28a7-4ac1-b96c-9dc56553d0da None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:37.342539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:37.343135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:37.343501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:37.343706 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] PATH_INFO: `/v3/projects/e861409abbf4431da49394f68330f5c0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:37.344883 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e861409abbf4431da49394f68330f5c0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:37.346854 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:37.401285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e861409abbf4431da49394f68330f5c0', 'request_id': 'req-b8976467-aa8e-4997-aa07-108d13ca2997'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.407852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e861409abbf4431da49394f68330f5c0', 'request_id': 'req-b8976467-aa8e-4997-aa07-108d13ca2997'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.415366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e861409abbf4431da49394f68330f5c0', 'request_id': 'req-b8976467-aa8e-4997-aa07-108d13ca2997'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.429627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] The token cache is being invalidate because project e861409abbf4431da49394f68330f5c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:37.429921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.430564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.432364 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1071/2150] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:37 2022] DELETE /identity/v3/projects/e861409abbf4431da49394f68330f5c0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:37.436297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a756903-c553-4bcd-8816-e68664fb84ea service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:37.446029 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a756903-c553-4bcd-8816-e68664fb84ea service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:37.455211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:37.508528 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:37.509160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:37.509437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:37.509737 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] PATH_INFO: `/v3/users/4384f763c6144904a9fd7d71bad28b2f` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:37.510889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4384f763c6144904a9fd7d71bad28b2f)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:37.512842 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:37.563222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4384f763c6144904a9fd7d71bad28b2f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.571668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4384f763c6144904a9fd7d71bad28b2f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.572457 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4384f763c6144904a9fd7d71bad28b2f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.586244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4384f763c6144904a9fd7d71bad28b2f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.602702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-010a8406-37c9-41d5-b875-6b800d4e3d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4384f763c6144904a9fd7d71bad28b2f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:37.609515 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1080/2151] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:37 2022] DELETE /identity/v3/users/4384f763c6144904a9fd7d71bad28b2f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:37.897423 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:37.899186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:37.915122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8976467-aa8e-4997-aa07-108d13ca2997 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:37.984369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:37.985007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:37.985285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:37.985557 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] PATH_INFO: `/v3/projects/71952745078747da8719384c8faf5ecd` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:37.986681 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71952745078747da8719384c8faf5ecd)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:37.988635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.037383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71952745078747da8719384c8faf5ecd', 'request_id': 'req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:38.042896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71952745078747da8719384c8faf5ecd', 'request_id': 'req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:38.049591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71952745078747da8719384c8faf5ecd', 'request_id': 'req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:38.063482 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] The token cache is being invalidate because project 71952745078747da8719384c8faf5ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:38.063832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:38.064451 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3bcd1365-6bf5-40b0-b747-8b6b8bae93c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:38.066163 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1072/2152] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:37 2022] DELETE /identity/v3/projects/71952745078747da8719384c8faf5ecd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:38.071102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbcacc-ebc7-4d9b-bf0c-4335a5b0004f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.071635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbcacc-ebc7-4d9b-bf0c-4335a5b0004f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.071635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbcacc-ebc7-4d9b-bf0c-4335a5b0004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.093845 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b1fbcacc-ebc7-4d9b-bf0c-4335a5b0004f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:38.095086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b1fbcacc-ebc7-4d9b-bf0c-4335a5b0004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:38.154702 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1081/2153] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:38.162678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69191e-34af-4615-897c-628c098f4679 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.162996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69191e-34af-4615-897c-628c098f4679 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.163280 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69191e-34af-4615-897c-628c098f4679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.186850 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ca69191e-34af-4615-897c-628c098f4679 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:38.188100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ca69191e-34af-4615-897c-628c098f4679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:38.197288 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1073/2154] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:38.201573 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbcacc-ebc7-4d9b-bf0c-4335a5b0004f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.212271 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbcacc-ebc7-4d9b-bf0c-4335a5b0004f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.217701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c448-e0d1-422b-a2c8-d5378e58285e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.218370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c448-e0d1-422b-a2c8-d5378e58285e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.218760 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c448-e0d1-422b-a2c8-d5378e58285e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.219046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c448-e0d1-422b-a2c8-d5378e58285e admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.219738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788c448-e0d1-422b-a2c8-d5378e58285e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:55:38.220043 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788c448-e0d1-422b-a2c8-d5378e58285e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.221661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788c448-e0d1-422b-a2c8-d5378e58285e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.234140 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1082/2155] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:55:38.238354 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69191e-34af-4615-897c-628c098f4679 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.248335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69191e-34af-4615-897c-628c098f4679 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.254433 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b72cda-08b6-425a-857f-58d9acb3cdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.255096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b72cda-08b6-425a-857f-58d9acb3cdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.255370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b72cda-08b6-425a-857f-58d9acb3cdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.255644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b72cda-08b6-425a-857f-58d9acb3cdd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.256291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b72cda-08b6-425a-857f-58d9acb3cdd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.258225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b72cda-08b6-425a-857f-58d9acb3cdd0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.263929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:38.294783 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1074/2156] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:38 2022] POST /identity/v3/projects => generated 361 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:38.298676 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c448-e0d1-422b-a2c8-d5378e58285e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.309069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c448-e0d1-422b-a2c8-d5378e58285e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.322011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:38.370390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1809bbf-6d2d-411c-b3b4-62c05b53a1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.371095 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1809bbf-6d2d-411c-b3b4-62c05b53a1fe admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.371376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1809bbf-6d2d-411c-b3b4-62c05b53a1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.371691 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1809bbf-6d2d-411c-b3b4-62c05b53a1fe admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.372612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1809bbf-6d2d-411c-b3b4-62c05b53a1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.374487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1809bbf-6d2d-411c-b3b4-62c05b53a1fe admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.398200 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1083/2157] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:38 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:38.402941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b72cda-08b6-425a-857f-58d9acb3cdd0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.413228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b72cda-08b6-425a-857f-58d9acb3cdd0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.418277 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec793bbf-2c1a-4668-b1c9-fde3835f3be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.419036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec793bbf-2c1a-4668-b1c9-fde3835f3be6 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.419368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec793bbf-2c1a-4668-b1c9-fde3835f3be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.419648 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec793bbf-2c1a-4668-b1c9-fde3835f3be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.420319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec793bbf-2c1a-4668-b1c9-fde3835f3be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.421999 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec793bbf-2c1a-4668-b1c9-fde3835f3be6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.433285 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1075/2158] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:38.437404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1809bbf-6d2d-411c-b3b4-62c05b53a1fe admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.447447 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1809bbf-6d2d-411c-b3b4-62c05b53a1fe admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.452367 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fa3b5-5714-44e2-9e6c-1820c94fb695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.452995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fa3b5-5714-44e2-9e6c-1820c94fb695 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.453272 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fa3b5-5714-44e2-9e6c-1820c94fb695 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.453545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fa3b5-5714-44e2-9e6c-1820c94fb695 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.454212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fa3b5-5714-44e2-9e6c-1820c94fb695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.455901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fa3b5-5714-44e2-9e6c-1820c94fb695 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.467308 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1084/2159] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:38.471595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec793bbf-2c1a-4668-b1c9-fde3835f3be6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.481941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec793bbf-2c1a-4668-b1c9-fde3835f3be6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.486826 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d060fa-9a27-4a3d-9664-63346cb62951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.487473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d060fa-9a27-4a3d-9664-63346cb62951 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.487750 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d060fa-9a27-4a3d-9664-63346cb62951 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.488022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d060fa-9a27-4a3d-9664-63346cb62951 admin admin] PATH_INFO: `/v3/projects/8756e2712afc46c0915218767d4540a6/users/85f7b67b4c1b401ab085d9349b36bd87/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.502635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d060fa-9a27-4a3d-9664-63346cb62951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8756e2712afc46c0915218767d4540a6, user_id=85f7b67b4c1b401ab085d9349b36bd87, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.505292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d060fa-9a27-4a3d-9664-63346cb62951 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.519765 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1076/2160] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:38 2022] PUT /identity/v3/projects/8756e2712afc46c0915218767d4540a6/users/85f7b67b4c1b401ab085d9349b36bd87/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:38.531933 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b22021-ddca-4164-becb-d37c40fd9e51 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.532217 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b22021-ddca-4164-becb-d37c40fd9e51 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.532488 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b22021-ddca-4164-becb-d37c40fd9e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.563528 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-33b22021-ddca-4164-becb-d37c40fd9e51 None None] MFA Rules not processed for user `85f7b67b4c1b401ab085d9349b36bd87`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:38.564829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-33b22021-ddca-4164-becb-d37c40fd9e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:38.633452 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1085/2161] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:38.638266 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d060fa-9a27-4a3d-9664-63346cb62951 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.704186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d060fa-9a27-4a3d-9664-63346cb62951 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.709761 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b2ecd-e6a0-4244-9dac-7c151360874c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.710414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b2ecd-e6a0-4244-9dac-7c151360874c admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.710700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b2ecd-e6a0-4244-9dac-7c151360874c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.711060 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b2ecd-e6a0-4244-9dac-7c151360874c admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.711673 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b2ecd-e6a0-4244-9dac-7c151360874c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.713668 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b2ecd-e6a0-4244-9dac-7c151360874c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.746709 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1077/2162] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:38 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:38.747887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b22021-ddca-4164-becb-d37c40fd9e51 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.759269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b22021-ddca-4164-becb-d37c40fd9e51 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.820234 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506130ae-02c6-4ba6-8d33-1f42667d38f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.820911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506130ae-02c6-4ba6-8d33-1f42667d38f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.821242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506130ae-02c6-4ba6-8d33-1f42667d38f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.821518 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506130ae-02c6-4ba6-8d33-1f42667d38f7 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.822405 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506130ae-02c6-4ba6-8d33-1f42667d38f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.824422 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506130ae-02c6-4ba6-8d33-1f42667d38f7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.853407 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1086/2163] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:38 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:55:38.857441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b2ecd-e6a0-4244-9dac-7c151360874c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.868307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b2ecd-e6a0-4244-9dac-7c151360874c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.872958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9e0af-5abd-4f42-918a-d7f74feec129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.873552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e0af-5abd-4f42-918a-d7f74feec129 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.873829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e0af-5abd-4f42-918a-d7f74feec129 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.874182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9e0af-5abd-4f42-918a-d7f74feec129 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.874767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9e0af-5abd-4f42-918a-d7f74feec129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.876401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9e0af-5abd-4f42-918a-d7f74feec129 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.890890 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1078/2164] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:55:38.894875 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506130ae-02c6-4ba6-8d33-1f42667d38f7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.905145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506130ae-02c6-4ba6-8d33-1f42667d38f7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:38.911851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ec925-f86b-439b-80e9-91b1a8d3bbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:38.912533 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ec925-f86b-439b-80e9-91b1a8d3bbc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:38.912803 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ec925-f86b-439b-80e9-91b1a8d3bbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:38.913071 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ec925-f86b-439b-80e9-91b1a8d3bbc2 admin admin] PATH_INFO: `/v3/projects/675467183c564f2094f73c1818436cc7/users/6b66e9142b184dd0ba6ff8beee1478ac/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:38.926611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:38.930401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ec925-f86b-439b-80e9-91b1a8d3bbc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=675467183c564f2094f73c1818436cc7, user_id=6b66e9142b184dd0ba6ff8beee1478ac, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:38.933220 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ec925-f86b-439b-80e9-91b1a8d3bbc2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:38.953029 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1087/2165] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:38 2022] PUT /identity/v3/projects/675467183c564f2094f73c1818436cc7/users/6b66e9142b184dd0ba6ff8beee1478ac/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:38.957351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9e0af-5abd-4f42-918a-d7f74feec129 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:38.974235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9e0af-5abd-4f42-918a-d7f74feec129 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:39.044804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d919f-c89f-4d3a-9299-10e373689cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:39.045419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d919f-c89f-4d3a-9299-10e373689cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:39.045693 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d919f-c89f-4d3a-9299-10e373689cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:39.045967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d919f-c89f-4d3a-9299-10e373689cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:39.046606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d919f-c89f-4d3a-9299-10e373689cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:39.049275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d919f-c89f-4d3a-9299-10e373689cd1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:39.063109 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1079/2166] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:39.067155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ec925-f86b-439b-80e9-91b1a8d3bbc2 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:39.077360 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ec925-f86b-439b-80e9-91b1a8d3bbc2 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:39.082311 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74488ec5-df07-4c0c-8e5c-08f90d976b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:39.083077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74488ec5-df07-4c0c-8e5c-08f90d976b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:39.083426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74488ec5-df07-4c0c-8e5c-08f90d976b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:39.083782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74488ec5-df07-4c0c-8e5c-08f90d976b11 admin admin] PATH_INFO: `/v3/projects/675467183c564f2094f73c1818436cc7/users/6b66e9142b184dd0ba6ff8beee1478ac/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:39.085811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74488ec5-df07-4c0c-8e5c-08f90d976b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=675467183c564f2094f73c1818436cc7, user_id=6b66e9142b184dd0ba6ff8beee1478ac, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:39.088755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74488ec5-df07-4c0c-8e5c-08f90d976b11 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:39.100763 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1088/2167] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:39 2022] PUT /identity/v3/projects/675467183c564f2094f73c1818436cc7/users/6b66e9142b184dd0ba6ff8beee1478ac/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:55:39.112392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd72848-6a5a-44fc-a013-e031f5c2eb4b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:39.112807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd72848-6a5a-44fc-a013-e031f5c2eb4b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:39.113112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd72848-6a5a-44fc-a013-e031f5c2eb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:39.146263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0dd72848-6a5a-44fc-a013-e031f5c2eb4b None None] MFA Rules not processed for user `6b66e9142b184dd0ba6ff8beee1478ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:39.149040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0dd72848-6a5a-44fc-a013-e031f5c2eb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:39.158436 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:39.223049 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1080/2168] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:55:39.234280 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74488ec5-df07-4c0c-8e5c-08f90d976b11 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:39.236111 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-74488ec5-df07-4c0c-8e5c-08f90d976b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:39.309874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74488ec5-df07-4c0c-8e5c-08f90d976b11 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:39.313885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a176a66-7fd5-4f3f-b9f2-413e4c997d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:39.314470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a176a66-7fd5-4f3f-b9f2-413e4c997d35 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:39.314765 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a176a66-7fd5-4f3f-b9f2-413e4c997d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:39.315139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a176a66-7fd5-4f3f-b9f2-413e4c997d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:39.326200 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a176a66-7fd5-4f3f-b9f2-413e4c997d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:39.327701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a176a66-7fd5-4f3f-b9f2-413e4c997d35 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:39.338306 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1089/2169] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:39.416109 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd72848-6a5a-44fc-a013-e031f5c2eb4b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:39.428844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd72848-6a5a-44fc-a013-e031f5c2eb4b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:39.432558 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd6770-72d2-441b-94a3-85dc2ba80360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:39.433170 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd6770-72d2-441b-94a3-85dc2ba80360 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:39.433441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd6770-72d2-441b-94a3-85dc2ba80360 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:39.433707 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd6770-72d2-441b-94a3-85dc2ba80360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:39.493510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd6770-72d2-441b-94a3-85dc2ba80360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:39.495007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd6770-72d2-441b-94a3-85dc2ba80360 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:39.507339 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1081/2170] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:40.222772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a176a66-7fd5-4f3f-b9f2-413e4c997d35 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:40.224308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7a176a66-7fd5-4f3f-b9f2-413e4c997d35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:40.319126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a176a66-7fd5-4f3f-b9f2-413e4c997d35 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:40.324476 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febe1fc-e33b-4dcd-b6c5-2de5f7a2cd99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:40.325086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febe1fc-e33b-4dcd-b6c5-2de5f7a2cd99 service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:40.325356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febe1fc-e33b-4dcd-b6c5-2de5f7a2cd99 service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:40.325684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febe1fc-e33b-4dcd-b6c5-2de5f7a2cd99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:40.327425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2febe1fc-e33b-4dcd-b6c5-2de5f7a2cd99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:40.339785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febe1fc-e33b-4dcd-b6c5-2de5f7a2cd99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:40.341384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febe1fc-e33b-4dcd-b6c5-2de5f7a2cd99 service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:40.351555 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1090/2171] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:42.310025 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd6770-72d2-441b-94a3-85dc2ba80360 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:42.381871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd6770-72d2-441b-94a3-85dc2ba80360 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:42.387393 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:42.388146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:42.388421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:42.388801 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] PATH_INFO: `/v3/users/85f7b67b4c1b401ab085d9349b36bd87` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:42.390081 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f7b67b4c1b401ab085d9349b36bd87)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:42.392016 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:42.453494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f7b67b4c1b401ab085d9349b36bd87'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:42.454181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f7b67b4c1b401ab085d9349b36bd87'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:42.455617 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:42.464195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f7b67b4c1b401ab085d9349b36bd87'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:42.473570 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f7b67b4c1b401ab085d9349b36bd87'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:42.494780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f7b67b4c1b401ab085d9349b36bd87'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:42.505214 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1082/2172] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:42 2022] DELETE /identity/v3/users/85f7b67b4c1b401ab085d9349b36bd87 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:42.510848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079a5f0-1da3-499e-81f2-562ea3b98bc9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:42.511554 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079a5f0-1da3-499e-81f2-562ea3b98bc9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:42.511990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079a5f0-1da3-499e-81f2-562ea3b98bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:42.535098 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3079a5f0-1da3-499e-81f2-562ea3b98bc9 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:42.536383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3079a5f0-1da3-499e-81f2-562ea3b98bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:42.590229 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:42.609461 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1091/2173] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:55:42.620524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:42.622330 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:42.688273 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bb46e-d2d8-4c7e-a662-e61a5ec79ac9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:42.693007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05f39-5cfa-40bb-a2a4-1d43e099ecb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:42.693618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05f39-5cfa-40bb-a2a4-1d43e099ecb0 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:42.693923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05f39-5cfa-40bb-a2a4-1d43e099ecb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:42.694195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05f39-5cfa-40bb-a2a4-1d43e099ecb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:42.712290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd05f39-5cfa-40bb-a2a4-1d43e099ecb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:42.713776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd05f39-5cfa-40bb-a2a4-1d43e099ecb0 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:42.724419 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1083/2174] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:43.010930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079a5f0-1da3-499e-81f2-562ea3b98bc9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:43.012377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3079a5f0-1da3-499e-81f2-562ea3b98bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:43.027831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079a5f0-1da3-499e-81f2-562ea3b98bc9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:43.033549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:43.034190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:43.034466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:43.034759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] PATH_INFO: `/v3/projects/8756e2712afc46c0915218767d4540a6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:43.035983 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8756e2712afc46c0915218767d4540a6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:43.038002 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:43.087252 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8756e2712afc46c0915218767d4540a6', 'request_id': 'req-50ff3dee-f376-455f-a601-4676874b55e8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.093028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8756e2712afc46c0915218767d4540a6', 'request_id': 'req-50ff3dee-f376-455f-a601-4676874b55e8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.099333 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8756e2712afc46c0915218767d4540a6', 'request_id': 'req-50ff3dee-f376-455f-a601-4676874b55e8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.114225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] The token cache is being invalidate because project 8756e2712afc46c0915218767d4540a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:43.114511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.115183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-50ff3dee-f376-455f-a601-4676874b55e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.119860 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1092/2175] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:43 2022] DELETE /identity/v3/projects/8756e2712afc46c0915218767d4540a6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:43.121150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05f39-5cfa-40bb-a2a4-1d43e099ecb0 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:43.131289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05f39-5cfa-40bb-a2a4-1d43e099ecb0 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:43.192113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:43.192733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:43.193033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:43.193289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] PATH_INFO: `/v3/users/6b66e9142b184dd0ba6ff8beee1478ac` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:43.194424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b66e9142b184dd0ba6ff8beee1478ac)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:43.196365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:43.248608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b66e9142b184dd0ba6ff8beee1478ac'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.249353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b66e9142b184dd0ba6ff8beee1478ac'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.257270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b66e9142b184dd0ba6ff8beee1478ac'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.273508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b66e9142b184dd0ba6ff8beee1478ac'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.285068 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:43.300918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b66e9142b184dd0ba6ff8beee1478ac'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.307203 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1084/2176] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:55:43 2022] DELETE /identity/v3/users/6b66e9142b184dd0ba6ff8beee1478ac => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:43.391203 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55592e26-6b25-43d9-9cc0-2e2da8d4f93b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:43.391488 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55592e26-6b25-43d9-9cc0-2e2da8d4f93b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:43.391759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55592e26-6b25-43d9-9cc0-2e2da8d4f93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:43.422090 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-55592e26-6b25-43d9-9cc0-2e2da8d4f93b None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:43.424869 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-55592e26-6b25-43d9-9cc0-2e2da8d4f93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:43.488059 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1093/2177] 10.222.0.26 () {58 vars in 967 bytes} [Mon Oct 24 08:55:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:43.503230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:43.504766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:43.578454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacff24-aae7-4668-bd87-c3f73b4e3dc6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:43.586219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab5898-d01d-4b3d-bbe7-6df491f53a5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:43.587132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab5898-d01d-4b3d-bbe7-6df491f53a5d service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:43.587407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab5898-d01d-4b3d-bbe7-6df491f53a5d service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:43.587725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab5898-d01d-4b3d-bbe7-6df491f53a5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:43.597453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab5898-d01d-4b3d-bbe7-6df491f53a5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:43.599227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab5898-d01d-4b3d-bbe7-6df491f53a5d service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:43.605435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55592e26-6b25-43d9-9cc0-2e2da8d4f93b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:43.605435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-55592e26-6b25-43d9-9cc0-2e2da8d4f93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:43.610482 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1085/2178] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:55:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:55:43.697040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55592e26-6b25-43d9-9cc0-2e2da8d4f93b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:43.709910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:43.710643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:43.710954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:43.711241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] PATH_INFO: `/v3/projects/675467183c564f2094f73c1818436cc7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:43.712575 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=675467183c564f2094f73c1818436cc7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:43.714909 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:43.729701 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:43.772605 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '675467183c564f2094f73c1818436cc7', 'request_id': 'req-3f78becc-58a8-4155-a66c-9a39d46422a5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.779271 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '675467183c564f2094f73c1818436cc7', 'request_id': 'req-3f78becc-58a8-4155-a66c-9a39d46422a5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.786359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '675467183c564f2094f73c1818436cc7', 'request_id': 'req-3f78becc-58a8-4155-a66c-9a39d46422a5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.805304 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] The token cache is being invalidate because project 675467183c564f2094f73c1818436cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:55:43.805746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.807030 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3f78becc-58a8-4155-a66c-9a39d46422a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:55:43.808657 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1094/2179] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:55:43 2022] DELETE /identity/v3/projects/675467183c564f2094f73c1818436cc7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:55:43.814015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862743c3-5792-4928-92fc-e5e14f12e023 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:43.814409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862743c3-5792-4928-92fc-e5e14f12e023 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:43.814781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862743c3-5792-4928-92fc-e5e14f12e023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:43.839924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-862743c3-5792-4928-92fc-e5e14f12e023 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:43.844350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-862743c3-5792-4928-92fc-e5e14f12e023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:43.932650 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1086/2180] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:43.943563 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd823f-1aa9-466f-85e3-5511889b46f0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:43.943822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd823f-1aa9-466f-85e3-5511889b46f0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:43.944102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd823f-1aa9-466f-85e3-5511889b46f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:43.965602 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-84bd823f-1aa9-466f-85e3-5511889b46f0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:43.967556 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-84bd823f-1aa9-466f-85e3-5511889b46f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:43.979084 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1095/2181] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:43.981405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862743c3-5792-4928-92fc-e5e14f12e023 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:43.992025 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862743c3-5792-4928-92fc-e5e14f12e023 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:43.997229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd916654-b706-4ffd-8c11-3727b4e78980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.000853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd916654-b706-4ffd-8c11-3727b4e78980 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.000853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd916654-b706-4ffd-8c11-3727b4e78980 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.000853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd916654-b706-4ffd-8c11-3727b4e78980 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.000853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd916654-b706-4ffd-8c11-3727b4e78980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:55:44.000853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd916654-b706-4ffd-8c11-3727b4e78980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.001665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd916654-b706-4ffd-8c11-3727b4e78980 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.023424 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1087/2182] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:55:44.024923 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd823f-1aa9-466f-85e3-5511889b46f0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.034496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd823f-1aa9-466f-85e3-5511889b46f0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.039404 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72eb5-a5b8-4a11-b6b2-c22d1ee16538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.045296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72eb5-a5b8-4a11-b6b2-c22d1ee16538 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.049177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72eb5-a5b8-4a11-b6b2-c22d1ee16538 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.049651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e72eb5-a5b8-4a11-b6b2-c22d1ee16538 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.050413 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e72eb5-a5b8-4a11-b6b2-c22d1ee16538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.052557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e72eb5-a5b8-4a11-b6b2-c22d1ee16538 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.086233 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1096/2183] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:44 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:44.087757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd916654-b706-4ffd-8c11-3727b4e78980 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.100457 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd916654-b706-4ffd-8c11-3727b4e78980 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.171522 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe47c2a-bb1d-4720-aeda-5201e8d146b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.172788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe47c2a-bb1d-4720-aeda-5201e8d146b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.173291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe47c2a-bb1d-4720-aeda-5201e8d146b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.173747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe47c2a-bb1d-4720-aeda-5201e8d146b6 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.174577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe47c2a-bb1d-4720-aeda-5201e8d146b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.176704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe47c2a-bb1d-4720-aeda-5201e8d146b6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.188908 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:44.206841 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1088/2184] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:44 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:44.210903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72eb5-a5b8-4a11-b6b2-c22d1ee16538 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.222325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e72eb5-a5b8-4a11-b6b2-c22d1ee16538 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.227733 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac306691-1b9e-43df-835f-147abd7191cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.228565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac306691-1b9e-43df-835f-147abd7191cf admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.228714 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac306691-1b9e-43df-835f-147abd7191cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.228879 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac306691-1b9e-43df-835f-147abd7191cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.229495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac306691-1b9e-43df-835f-147abd7191cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.231332 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac306691-1b9e-43df-835f-147abd7191cf admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.246141 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1097/2185] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:44.250811 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe47c2a-bb1d-4720-aeda-5201e8d146b6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.261975 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe47c2a-bb1d-4720-aeda-5201e8d146b6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.269425 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd048113-a835-4d59-ba4e-54066ee958ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.270203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd048113-a835-4d59-ba4e-54066ee958ab admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.270489 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd048113-a835-4d59-ba4e-54066ee958ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.270842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd048113-a835-4d59-ba4e-54066ee958ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.271631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd048113-a835-4d59-ba4e-54066ee958ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.274124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd048113-a835-4d59-ba4e-54066ee958ab admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.286025 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1089/2186] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:55:44.291083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac306691-1b9e-43df-835f-147abd7191cf admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.301502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac306691-1b9e-43df-835f-147abd7191cf admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.317621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81393f1f-0f11-4829-8551-e3d375e4f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.317621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81393f1f-0f11-4829-8551-e3d375e4f4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.317621 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81393f1f-0f11-4829-8551-e3d375e4f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.318004 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81393f1f-0f11-4829-8551-e3d375e4f4f5 admin admin] PATH_INFO: `/v3/projects/83c9b02c3d254be68af3afde96d4b61b/users/71aa7b946c2b44b29d7df4e9e84467a9/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.332911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81393f1f-0f11-4829-8551-e3d375e4f4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83c9b02c3d254be68af3afde96d4b61b, user_id=71aa7b946c2b44b29d7df4e9e84467a9, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.335580 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81393f1f-0f11-4829-8551-e3d375e4f4f5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.354766 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1098/2187] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:44 2022] PUT /identity/v3/projects/83c9b02c3d254be68af3afde96d4b61b/users/71aa7b946c2b44b29d7df4e9e84467a9/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:44.365218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72c0b4-6b67-47f6-b075-9e8ac9720e4d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.365591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72c0b4-6b67-47f6-b075-9e8ac9720e4d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.365937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72c0b4-6b67-47f6-b075-9e8ac9720e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.395356 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8e72c0b4-6b67-47f6-b075-9e8ac9720e4d None None] MFA Rules not processed for user `71aa7b946c2b44b29d7df4e9e84467a9`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:44.396641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8e72c0b4-6b67-47f6-b075-9e8ac9720e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:44.464293 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1090/2188] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:44.469102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81393f1f-0f11-4829-8551-e3d375e4f4f5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.535771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81393f1f-0f11-4829-8551-e3d375e4f4f5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.541081 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb41ee-aa3a-4235-aadc-621ba95ee644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.541700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb41ee-aa3a-4235-aadc-621ba95ee644 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.542011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb41ee-aa3a-4235-aadc-621ba95ee644 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.542281 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb41ee-aa3a-4235-aadc-621ba95ee644 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.542964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb41ee-aa3a-4235-aadc-621ba95ee644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.544883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb41ee-aa3a-4235-aadc-621ba95ee644 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.577258 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1099/2189] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:55:44 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:44.578453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72c0b4-6b67-47f6-b075-9e8ac9720e4d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.590965 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72c0b4-6b67-47f6-b075-9e8ac9720e4d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.651484 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed407a2-3669-4129-be4e-4c8fe8262ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.652099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed407a2-3669-4129-be4e-4c8fe8262ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.652375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed407a2-3669-4129-be4e-4c8fe8262ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.652642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed407a2-3669-4129-be4e-4c8fe8262ca3 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.653272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed407a2-3669-4129-be4e-4c8fe8262ca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.655285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed407a2-3669-4129-be4e-4c8fe8262ca3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.683570 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1091/2190] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:55:44 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:55:44.684741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb41ee-aa3a-4235-aadc-621ba95ee644 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.696752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb41ee-aa3a-4235-aadc-621ba95ee644 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.702108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faf36c-42fe-4874-b1e0-4c2ce8fe0300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.702730 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faf36c-42fe-4874-b1e0-4c2ce8fe0300 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.703024 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faf36c-42fe-4874-b1e0-4c2ce8fe0300 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.703294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faf36c-42fe-4874-b1e0-4c2ce8fe0300 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.703937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63faf36c-42fe-4874-b1e0-4c2ce8fe0300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.705561 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63faf36c-42fe-4874-b1e0-4c2ce8fe0300 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.717035 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1100/2191] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:44.721378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed407a2-3669-4129-be4e-4c8fe8262ca3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.731776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed407a2-3669-4129-be4e-4c8fe8262ca3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.736970 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5346f9-467a-450b-a2d6-380c8e3b165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.737614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5346f9-467a-450b-a2d6-380c8e3b165e admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.737892 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5346f9-467a-450b-a2d6-380c8e3b165e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.738160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5346f9-467a-450b-a2d6-380c8e3b165e admin admin] PATH_INFO: `/v3/projects/1e49dc8bc8b04e74947d42dd08e382ee/users/4bd0bd1df75943db92c5513f72b010f4/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.754889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5346f9-467a-450b-a2d6-380c8e3b165e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e49dc8bc8b04e74947d42dd08e382ee, user_id=4bd0bd1df75943db92c5513f72b010f4, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.757539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5346f9-467a-450b-a2d6-380c8e3b165e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.772618 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1092/2192] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:44 2022] PUT /identity/v3/projects/1e49dc8bc8b04e74947d42dd08e382ee/users/4bd0bd1df75943db92c5513f72b010f4/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:44.777253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faf36c-42fe-4874-b1e0-4c2ce8fe0300 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.787757 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63faf36c-42fe-4874-b1e0-4c2ce8fe0300 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.796026 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:44.845498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab64b3-91c8-451e-bdd2-6d9a38ced078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.846089 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab64b3-91c8-451e-bdd2-6d9a38ced078 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.846362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab64b3-91c8-451e-bdd2-6d9a38ced078 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.846637 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab64b3-91c8-451e-bdd2-6d9a38ced078 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.847293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab64b3-91c8-451e-bdd2-6d9a38ced078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.848873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab64b3-91c8-451e-bdd2-6d9a38ced078 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.858536 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1101/2193] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:55:44.862833 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5346f9-467a-450b-a2d6-380c8e3b165e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:44.873591 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5346f9-467a-450b-a2d6-380c8e3b165e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:44.878839 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5285d-a8ee-45ee-abae-e2c036277810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:44.879478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5285d-a8ee-45ee-abae-e2c036277810 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.879755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5285d-a8ee-45ee-abae-e2c036277810 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.880031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5285d-a8ee-45ee-abae-e2c036277810 admin admin] PATH_INFO: `/v3/projects/1e49dc8bc8b04e74947d42dd08e382ee/users/4bd0bd1df75943db92c5513f72b010f4/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.881985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5285d-a8ee-45ee-abae-e2c036277810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e49dc8bc8b04e74947d42dd08e382ee, user_id=4bd0bd1df75943db92c5513f72b010f4, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:44.884613 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5285d-a8ee-45ee-abae-e2c036277810 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:44.895120 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1093/2194] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:55:44 2022] PUT /identity/v3/projects/1e49dc8bc8b04e74947d42dd08e382ee/users/4bd0bd1df75943db92c5513f72b010f4/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:55:44.905392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64041f48-dff7-43f6-861f-8f6a13c99c25 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:44.905675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64041f48-dff7-43f6-861f-8f6a13c99c25 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:44.905946 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64041f48-dff7-43f6-861f-8f6a13c99c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:44.939232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-64041f48-dff7-43f6-861f-8f6a13c99c25 None None] MFA Rules not processed for user `4bd0bd1df75943db92c5513f72b010f4`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:55:44.940453 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-64041f48-dff7-43f6-861f-8f6a13c99c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:45.008411 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1102/2195] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:55:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:55:45.019620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5285d-a8ee-45ee-abae-e2c036277810 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:45.021527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-71f5285d-a8ee-45ee-abae-e2c036277810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:45.050192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:45.080963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5285d-a8ee-45ee-abae-e2c036277810 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:45.084794 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999fe0a-54e2-4db7-83cb-0628b7504353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:45.085396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999fe0a-54e2-4db7-83cb-0628b7504353 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:45.085676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999fe0a-54e2-4db7-83cb-0628b7504353 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:45.085986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999fe0a-54e2-4db7-83cb-0628b7504353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:45.143501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999fe0a-54e2-4db7-83cb-0628b7504353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:45.145116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999fe0a-54e2-4db7-83cb-0628b7504353 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:45.155073 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1094/2196] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:48.812770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64041f48-dff7-43f6-861f-8f6a13c99c25 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:55:48.814409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-64041f48-dff7-43f6-861f-8f6a13c99c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:55:48.827424 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64041f48-dff7-43f6-861f-8f6a13c99c25 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:55:48.831280 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd75623-057c-46de-a195-fa4366215a25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:55:48.831846 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd75623-057c-46de-a195-fa4366215a25 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:55:48.832162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd75623-057c-46de-a195-fa4366215a25 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:55:48.832435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd75623-057c-46de-a195-fa4366215a25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:55:48.842202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd75623-057c-46de-a195-fa4366215a25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:55:48.843668 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd75623-057c-46de-a195-fa4366215a25 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:55:48.853443 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1103/2197] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:55:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:55:55.150703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=37 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:55:58.849166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=64 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:01.509927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999fe0a-54e2-4db7-83cb-0628b7504353 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:01.574954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999fe0a-54e2-4db7-83cb-0628b7504353 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:01.580039 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:01.580673 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:01.580951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:01.581223 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] PATH_INFO: `/v3/users/71aa7b946c2b44b29d7df4e9e84467a9` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:01.582380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=71aa7b946c2b44b29d7df4e9e84467a9)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:01.584366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:01.631909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71aa7b946c2b44b29d7df4e9e84467a9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:01.632549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71aa7b946c2b44b29d7df4e9e84467a9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:01.639900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71aa7b946c2b44b29d7df4e9e84467a9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:01.648416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71aa7b946c2b44b29d7df4e9e84467a9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:01.660526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71aa7b946c2b44b29d7df4e9e84467a9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:01.666404 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1095/2198] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:01 2022] DELETE /identity/v3/users/71aa7b946c2b44b29d7df4e9e84467a9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:01.670992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95077695-8db0-419a-ba46-75cfc66197c6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:01.671488 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95077695-8db0-419a-ba46-75cfc66197c6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:01.671587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95077695-8db0-419a-ba46-75cfc66197c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:01.690956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-95077695-8db0-419a-ba46-75cfc66197c6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:01.692161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-95077695-8db0-419a-ba46-75cfc66197c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:01.761482 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1104/2199] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:01.775494 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:01.778060 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:01.866206 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da24abf-e8c9-4cfe-bcc1-711c4ba5ba2a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:01.872658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691f611-616e-4284-aabd-dde02a153821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:01.872658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691f611-616e-4284-aabd-dde02a153821 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:01.872897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691f611-616e-4284-aabd-dde02a153821 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:01.873378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691f611-616e-4284-aabd-dde02a153821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:01.883826 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691f611-616e-4284-aabd-dde02a153821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:01.885760 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691f611-616e-4284-aabd-dde02a153821 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:01.896114 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1096/2200] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:02.173783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95077695-8db0-419a-ba46-75cfc66197c6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:02.175232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-95077695-8db0-419a-ba46-75cfc66197c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:02.190407 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95077695-8db0-419a-ba46-75cfc66197c6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:02.196751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:02.197572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:02.197852 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:02.198134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] PATH_INFO: `/v3/projects/83c9b02c3d254be68af3afde96d4b61b` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:02.199856 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] RBAC: Authorizing `identity:delete_project(project_id=83c9b02c3d254be68af3afde96d4b61b)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:02.201843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:02.248636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83c9b02c3d254be68af3afde96d4b61b', 'request_id': 'req-52819ee4-8937-48c6-9980-36ee47ccb92e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.254031 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83c9b02c3d254be68af3afde96d4b61b', 'request_id': 'req-52819ee4-8937-48c6-9980-36ee47ccb92e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.260766 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83c9b02c3d254be68af3afde96d4b61b', 'request_id': 'req-52819ee4-8937-48c6-9980-36ee47ccb92e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.276733 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] The token cache is being invalidate because project 83c9b02c3d254be68af3afde96d4b61b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:02.277144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.277775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.279689 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1105/2201] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:02 2022] DELETE /identity/v3/projects/83c9b02c3d254be68af3afde96d4b61b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:02.283592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691f611-616e-4284-aabd-dde02a153821 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:02.288552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:02.295702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691f611-616e-4284-aabd-dde02a153821 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:02.352971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:02.353596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:02.353872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:02.354148 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] PATH_INFO: `/v3/users/4bd0bd1df75943db92c5513f72b010f4` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:02.355325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd0bd1df75943db92c5513f72b010f4)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:02.357236 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:02.404602 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd0bd1df75943db92c5513f72b010f4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.405268 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd0bd1df75943db92c5513f72b010f4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.412277 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd0bd1df75943db92c5513f72b010f4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.421039 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd0bd1df75943db92c5513f72b010f4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.433988 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-113a1ece-dbb6-45a0-b3b5-248cbcf355b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd0bd1df75943db92c5513f72b010f4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.439895 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1097/2202] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:02 2022] DELETE /identity/v3/users/4bd0bd1df75943db92c5513f72b010f4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:02.716735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:02.718255 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:02.733567 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52819ee4-8937-48c6-9980-36ee47ccb92e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:02.791032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:02.791691 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:02.791970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:02.792242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] PATH_INFO: `/v3/projects/1e49dc8bc8b04e74947d42dd08e382ee` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:02.793399 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e49dc8bc8b04e74947d42dd08e382ee)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:02.795448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:02.828700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:02.846348 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e49dc8bc8b04e74947d42dd08e382ee', 'request_id': 'req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.852014 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e49dc8bc8b04e74947d42dd08e382ee', 'request_id': 'req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.858152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e49dc8bc8b04e74947d42dd08e382ee', 'request_id': 'req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.871427 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] The token cache is being invalidate because project 1e49dc8bc8b04e74947d42dd08e382ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:02.871713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.872370 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0cf2610-2bfe-4802-b714-ea6ce61f10ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:02.873854 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1106/2203] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:02 2022] DELETE /identity/v3/projects/1e49dc8bc8b04e74947d42dd08e382ee => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:02.879288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f66ff-f761-45d5-88d5-324b1a93b35f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:02.879818 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f66ff-f761-45d5-88d5-324b1a93b35f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:02.880205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f66ff-f761-45d5-88d5-324b1a93b35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:02.906355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e88f66ff-f761-45d5-88d5-324b1a93b35f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:02.908246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e88f66ff-f761-45d5-88d5-324b1a93b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:02.975035 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1098/2204] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:56:02.982934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d824b-6496-44ff-a77b-409d662cd5d0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:02.983218 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d824b-6496-44ff-a77b-409d662cd5d0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:02.983493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d824b-6496-44ff-a77b-409d662cd5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.004266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-779d824b-6496-44ff-a77b-409d662cd5d0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:03.005431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-779d824b-6496-44ff-a77b-409d662cd5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:03.015712 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1107/2205] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:03.021537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f66ff-f761-45d5-88d5-324b1a93b35f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.032029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f66ff-f761-45d5-88d5-324b1a93b35f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.037194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43f545-08e5-4fcd-80a1-5cb498e34ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.037799 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43f545-08e5-4fcd-80a1-5cb498e34ffa admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.038192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43f545-08e5-4fcd-80a1-5cb498e34ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.038601 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43f545-08e5-4fcd-80a1-5cb498e34ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.039303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43f545-08e5-4fcd-80a1-5cb498e34ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:03.039605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43f545-08e5-4fcd-80a1-5cb498e34ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.041303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43f545-08e5-4fcd-80a1-5cb498e34ffa admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.053190 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1099/2206] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:03.057547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d824b-6496-44ff-a77b-409d662cd5d0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.067357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d824b-6496-44ff-a77b-409d662cd5d0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.073254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4bd61-97a1-4fbd-90f6-486b2953aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.073828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4bd61-97a1-4fbd-90f6-486b2953aaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.074132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4bd61-97a1-4fbd-90f6-486b2953aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.074399 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4bd61-97a1-4fbd-90f6-486b2953aaa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.075101 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4bd61-97a1-4fbd-90f6-486b2953aaa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.077097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4bd61-97a1-4fbd-90f6-486b2953aaa6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.111592 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1108/2207] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:03 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:03.120188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43f545-08e5-4fcd-80a1-5cb498e34ffa admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.131513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43f545-08e5-4fcd-80a1-5cb498e34ffa admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.136658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:03.196794 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b158a-a46f-4a0b-a496-cdec429e2e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.198088 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b158a-a46f-4a0b-a496-cdec429e2e5b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.198440 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b158a-a46f-4a0b-a496-cdec429e2e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.198784 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b158a-a46f-4a0b-a496-cdec429e2e5b admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.199705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b158a-a46f-4a0b-a496-cdec429e2e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.202672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b158a-a46f-4a0b-a496-cdec429e2e5b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.231521 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1100/2208] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:03 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:03.235759 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4bd61-97a1-4fbd-90f6-486b2953aaa6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.246545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4bd61-97a1-4fbd-90f6-486b2953aaa6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.251774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717918ce-ed78-4720-aef5-31809453281e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.252367 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717918ce-ed78-4720-aef5-31809453281e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.252644 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717918ce-ed78-4720-aef5-31809453281e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.252913 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717918ce-ed78-4720-aef5-31809453281e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.253555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717918ce-ed78-4720-aef5-31809453281e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.255222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717918ce-ed78-4720-aef5-31809453281e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.266357 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1109/2209] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:56:03.270595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b158a-a46f-4a0b-a496-cdec429e2e5b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.283625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b158a-a46f-4a0b-a496-cdec429e2e5b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.289332 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf39316-592b-448f-816b-54a91c1a472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.289957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf39316-592b-448f-816b-54a91c1a472b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.290237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf39316-592b-448f-816b-54a91c1a472b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.290510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf39316-592b-448f-816b-54a91c1a472b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.291212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf39316-592b-448f-816b-54a91c1a472b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.292845 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf39316-592b-448f-816b-54a91c1a472b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.304263 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1101/2210] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:03.308822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717918ce-ed78-4720-aef5-31809453281e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.319450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717918ce-ed78-4720-aef5-31809453281e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.325237 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbadb9-eaaf-4ce0-b232-a4fb51b0ba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.325899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbadb9-eaaf-4ce0-b232-a4fb51b0ba68 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.326176 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbadb9-eaaf-4ce0-b232-a4fb51b0ba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.326445 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbadb9-eaaf-4ce0-b232-a4fb51b0ba68 admin admin] PATH_INFO: `/v3/projects/97d02f8dc3124b13a0a64a39348f3fbf/users/f3519febbbd94c089387f1b6bcf76447/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.342023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbadb9-eaaf-4ce0-b232-a4fb51b0ba68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d02f8dc3124b13a0a64a39348f3fbf, user_id=f3519febbbd94c089387f1b6bcf76447, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.344781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbadb9-eaaf-4ce0-b232-a4fb51b0ba68 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.360056 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1110/2211] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:03 2022] PUT /identity/v3/projects/97d02f8dc3124b13a0a64a39348f3fbf/users/f3519febbbd94c089387f1b6bcf76447/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:03.370165 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5208311-f14b-4f57-8325-8de87789a928 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.370456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5208311-f14b-4f57-8325-8de87789a928 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.370745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5208311-f14b-4f57-8325-8de87789a928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.400534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e5208311-f14b-4f57-8325-8de87789a928 None None] MFA Rules not processed for user `f3519febbbd94c089387f1b6bcf76447`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:03.401730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e5208311-f14b-4f57-8325-8de87789a928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:03.468714 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1102/2212] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:03.473572 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbadb9-eaaf-4ce0-b232-a4fb51b0ba68 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.535896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbadb9-eaaf-4ce0-b232-a4fb51b0ba68 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.541169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999ab87-de22-4e20-8972-e435f6cf7a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.541794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999ab87-de22-4e20-8972-e435f6cf7a3e admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.542093 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999ab87-de22-4e20-8972-e435f6cf7a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.542359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999ab87-de22-4e20-8972-e435f6cf7a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.543011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999ab87-de22-4e20-8972-e435f6cf7a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.544995 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999ab87-de22-4e20-8972-e435f6cf7a3e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.573194 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1111/2213] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:03 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:03.577456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5208311-f14b-4f57-8325-8de87789a928 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.590355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5208311-f14b-4f57-8325-8de87789a928 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.661305 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189c8db-b3c5-4fbc-955f-c856a0c5945a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.661906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189c8db-b3c5-4fbc-955f-c856a0c5945a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.662183 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189c8db-b3c5-4fbc-955f-c856a0c5945a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.662450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189c8db-b3c5-4fbc-955f-c856a0c5945a admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.663123 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189c8db-b3c5-4fbc-955f-c856a0c5945a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.665015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189c8db-b3c5-4fbc-955f-c856a0c5945a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.690392 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1103/2214] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:03 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:03.695168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999ab87-de22-4e20-8972-e435f6cf7a3e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.707214 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999ab87-de22-4e20-8972-e435f6cf7a3e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.711951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fd0d8-bbde-4fc9-83a4-fa904b29a918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.712550 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fd0d8-bbde-4fc9-83a4-fa904b29a918 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.712824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fd0d8-bbde-4fc9-83a4-fa904b29a918 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.713097 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fd0d8-bbde-4fc9-83a4-fa904b29a918 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.713765 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fd0d8-bbde-4fc9-83a4-fa904b29a918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.715415 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fd0d8-bbde-4fc9-83a4-fa904b29a918 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.722809 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189c8db-b3c5-4fbc-955f-c856a0c5945a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.724568 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a189c8db-b3c5-4fbc-955f-c856a0c5945a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:03.729319 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1112/2215] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:03.730953 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fd0d8-bbde-4fc9-83a4-fa904b29a918 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.740539 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189c8db-b3c5-4fbc-955f-c856a0c5945a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.743657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fd0d8-bbde-4fc9-83a4-fa904b29a918 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.745665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.746358 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.746626 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.746933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] PATH_INFO: `/v3/users/31d493ed96b74a4e8d897dcea947ac0e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.748223 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31d493ed96b74a4e8d897dcea947ac0e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.750162 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.750783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6630fec-c466-4567-8eea-2de6b593d455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.751434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6630fec-c466-4567-8eea-2de6b593d455 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.751719 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6630fec-c466-4567-8eea-2de6b593d455 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.751981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:03.752104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6630fec-c466-4567-8eea-2de6b593d455 admin admin] PATH_INFO: `/v3/projects/7fa76decffc847abbcdab96a42e92e88/users/0a8495b01cbf46d8833ecf6d5563e4ec/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.768587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6630fec-c466-4567-8eea-2de6b593d455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa76decffc847abbcdab96a42e92e88, user_id=0a8495b01cbf46d8833ecf6d5563e4ec, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.771352 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6630fec-c466-4567-8eea-2de6b593d455 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.780894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:03.786259 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1113/2216] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:03 2022] PUT /identity/v3/projects/7fa76decffc847abbcdab96a42e92e88/users/0a8495b01cbf46d8833ecf6d5563e4ec/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:03.790966 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6630fec-c466-4567-8eea-2de6b593d455 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.803044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31d493ed96b74a4e8d897dcea947ac0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:03.803714 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31d493ed96b74a4e8d897dcea947ac0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:03.810747 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31d493ed96b74a4e8d897dcea947ac0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:03.819110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31d493ed96b74a4e8d897dcea947ac0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:03.831200 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a9add392-f5da-4544-af34-214758ee1fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31d493ed96b74a4e8d897dcea947ac0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:03.836970 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1104/2217] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:03 2022] DELETE /identity/v3/users/31d493ed96b74a4e8d897dcea947ac0e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:03.841470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ba0ae-6c0f-47c2-b329-9717a726a1b0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.841741 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ba0ae-6c0f-47c2-b329-9717a726a1b0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.842024 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ba0ae-6c0f-47c2-b329-9717a726a1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.854056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6630fec-c466-4567-8eea-2de6b593d455 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.861233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-900ba0ae-6c0f-47c2-b329-9717a726a1b0 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:03.862446 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-900ba0ae-6c0f-47c2-b329-9717a726a1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:03.911162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd825bb2-6355-4a0a-9ea0-930f0f8053d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.911786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd825bb2-6355-4a0a-9ea0-930f0f8053d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.912067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd825bb2-6355-4a0a-9ea0-930f0f8053d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.912345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd825bb2-6355-4a0a-9ea0-930f0f8053d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.913104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd825bb2-6355-4a0a-9ea0-930f0f8053d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.914791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd825bb2-6355-4a0a-9ea0-930f0f8053d6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.921854 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1105/2218] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:56:03.924984 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1114/2219] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:03.929195 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ba0ae-6c0f-47c2-b329-9717a726a1b0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.930983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-900ba0ae-6c0f-47c2-b329-9717a726a1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:03.931574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd825bb2-6355-4a0a-9ea0-930f0f8053d6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:03.933201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bd825bb2-6355-4a0a-9ea0-930f0f8053d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:03.943398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ba0ae-6c0f-47c2-b329-9717a726a1b0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:03.948370 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b6736-f158-44ac-835d-d18235365e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:03.949026 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b6736-f158-44ac-835d-d18235365e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.949321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b6736-f158-44ac-835d-d18235365e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.949577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b6736-f158-44ac-835d-d18235365e76 admin admin] PATH_INFO: `/v3/projects/7fa76decffc847abbcdab96a42e92e88/users/0a8495b01cbf46d8833ecf6d5563e4ec/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.951579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b6736-f158-44ac-835d-d18235365e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa76decffc847abbcdab96a42e92e88, user_id=0a8495b01cbf46d8833ecf6d5563e4ec, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:03.954270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b6736-f158-44ac-835d-d18235365e76 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:03.964623 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1106/2220] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:03 2022] PUT /identity/v3/projects/7fa76decffc847abbcdab96a42e92e88/users/0a8495b01cbf46d8833ecf6d5563e4ec/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:03.975058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5ae2a-6cbc-4506-a9ae-af0c1a2e6fe0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:03.975344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5ae2a-6cbc-4506-a9ae-af0c1a2e6fe0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:03.975616 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5ae2a-6cbc-4506-a9ae-af0c1a2e6fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:03.998966 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd825bb2-6355-4a0a-9ea0-930f0f8053d6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:04.003119 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0ee5ae2a-6cbc-4506-a9ae-af0c1a2e6fe0 None None] MFA Rules not processed for user `0a8495b01cbf46d8833ecf6d5563e4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:04.004327 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0ee5ae2a-6cbc-4506-a9ae-af0c1a2e6fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:04.048896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db901b48-6366-4be0-a6e2-6f51c3610008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:04.049595 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db901b48-6366-4be0-a6e2-6f51c3610008 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:04.049990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db901b48-6366-4be0-a6e2-6f51c3610008 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:04.050350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db901b48-6366-4be0-a6e2-6f51c3610008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:04.054572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:04.063023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:04.076642 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1107/2221] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:04.089237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5ae2a-6cbc-4506-a9ae-af0c1a2e6fe0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:04.090742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0ee5ae2a-6cbc-4506-a9ae-af0c1a2e6fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:04.119038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db901b48-6366-4be0-a6e2-6f51c3610008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:04.120488 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db901b48-6366-4be0-a6e2-6f51c3610008 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:04.130455 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1115/2222] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:04.170293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5ae2a-6cbc-4506-a9ae-af0c1a2e6fe0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:04.175026 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8a16e-38ad-41c7-a38a-b2aa107d9918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:04.175456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8a16e-38ad-41c7-a38a-b2aa107d9918 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:04.175841 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8a16e-38ad-41c7-a38a-b2aa107d9918 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:04.176228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8a16e-38ad-41c7-a38a-b2aa107d9918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:04.187043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8a16e-38ad-41c7-a38a-b2aa107d9918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:04.188475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8a16e-38ad-41c7-a38a-b2aa107d9918 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:04.199926 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1108/2223] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:56:04.408843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db901b48-6366-4be0-a6e2-6f51c3610008 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:04.410937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-db901b48-6366-4be0-a6e2-6f51c3610008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:04.426961 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db901b48-6366-4be0-a6e2-6f51c3610008 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:04.433735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:04.434385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:04.434666 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:04.435036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] PATH_INFO: `/v3/projects/48ae99be2f1944bb89c5291ecb6ee02a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:04.436251 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ae99be2f1944bb89c5291ecb6ee02a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:04.438165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:04.487247 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ae99be2f1944bb89c5291ecb6ee02a', 'request_id': 'req-5021cb7f-47be-468f-8299-b6dfe6d77a12'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.492973 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ae99be2f1944bb89c5291ecb6ee02a', 'request_id': 'req-5021cb7f-47be-468f-8299-b6dfe6d77a12'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.499346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ae99be2f1944bb89c5291ecb6ee02a', 'request_id': 'req-5021cb7f-47be-468f-8299-b6dfe6d77a12'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.515433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] The token cache is being invalidate because project 48ae99be2f1944bb89c5291ecb6ee02a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:04.515745 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.516369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.518836 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1116/2224] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:04 2022] DELETE /identity/v3/projects/48ae99be2f1944bb89c5291ecb6ee02a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:04.548216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8a16e-38ad-41c7-a38a-b2aa107d9918 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:04.557894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8a16e-38ad-41c7-a38a-b2aa107d9918 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:04.619705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f32853-cdac-4478-afa0-0c1f44b213ba tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a8495b01cbf46d8833ecf6d5563e4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fa76decffc847abbcdab96a42e92e88', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:04.620341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f32853-cdac-4478-afa0-0c1f44b213ba tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:04.620625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f32853-cdac-4478-afa0-0c1f44b213ba tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:04.620899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f32853-cdac-4478-afa0-0c1f44b213ba tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:04.621555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f32853-cdac-4478-afa0-0c1f44b213ba tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:04.621885 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f32853-cdac-4478-afa0-0c1f44b213ba tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:04.623636 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f32853-cdac-4478-afa0-0c1f44b213ba tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:04.636465 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1109/2225] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:04.640865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:04.642450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:04.656983 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021cb7f-47be-468f-8299-b6dfe6d77a12 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:04.661926 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74f691-bc48-46da-929a-2ede58c55886 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a8495b01cbf46d8833ecf6d5563e4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fa76decffc847abbcdab96a42e92e88', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:04.662559 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74f691-bc48-46da-929a-2ede58c55886 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:04.662897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74f691-bc48-46da-929a-2ede58c55886 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:04.663178 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74f691-bc48-46da-929a-2ede58c55886 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:04.663836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74f691-bc48-46da-929a-2ede58c55886 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:04.665804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74f691-bc48-46da-929a-2ede58c55886 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:04.699182 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1117/2226] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:04 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:04.700508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f32853-cdac-4478-afa0-0c1f44b213ba tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:04.711353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f32853-cdac-4478-afa0-0c1f44b213ba tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:04.780840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a8495b01cbf46d8833ecf6d5563e4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fa76decffc847abbcdab96a42e92e88', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:04.781450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:04.781726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:04.781998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:04.782637 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:04.782968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:04.784554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:04.796484 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1110/2227] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:04.833237 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74f691-bc48-46da-929a-2ede58c55886 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:04.844663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74f691-bc48-46da-929a-2ede58c55886 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:04.849786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a8495b01cbf46d8833ecf6d5563e4ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fa76decffc847abbcdab96a42e92e88', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:04.850433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:04.850750 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:04.851017 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] PATH_INFO: `/v3/projects/d579f0f660da418bbce0f32776cb8165` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:04.852194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d579f0f660da418bbce0f32776cb8165)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:04.854094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:04.904477 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd579f0f660da418bbce0f32776cb8165', 'request_id': 'req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.910169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd579f0f660da418bbce0f32776cb8165', 'request_id': 'req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.916946 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd579f0f660da418bbce0f32776cb8165', 'request_id': 'req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.919531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:04.933245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] The token cache is being invalidate because project d579f0f660da418bbce0f32776cb8165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:04.933557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.934177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9cde8ef2-6f99-491f-8cd0-e68e44588ed4 tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:04.935705 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1118/2228] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:04 2022] DELETE /identity/v3/projects/d579f0f660da418bbce0f32776cb8165 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:56:05.137141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:05.139212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:05.145153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:05.211518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f592a-7e1f-47d6-80d2-9aef9757f21c tempest-VolumeQuotaClassesTest-1691064378 tempest-VolumeQuotaClassesTest-1691064378-project-admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:05.270051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:05.270679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:05.270983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:05.271250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] PATH_INFO: `/v3/users/f3519febbbd94c089387f1b6bcf76447` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:05.272516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3519febbbd94c089387f1b6bcf76447)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:05.274472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:05.327613 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3519febbbd94c089387f1b6bcf76447'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.328341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3519febbbd94c089387f1b6bcf76447'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.335347 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3519febbbd94c089387f1b6bcf76447'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.345657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3519febbbd94c089387f1b6bcf76447'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.358132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3519febbbd94c089387f1b6bcf76447'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.364224 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1111/2229] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:05 2022] DELETE /identity/v3/users/f3519febbbd94c089387f1b6bcf76447 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:05.368782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c602b-0da5-4ef3-bfc2-df5ee9e62ca7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:05.369070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c602b-0da5-4ef3-bfc2-df5ee9e62ca7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:05.369344 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c602b-0da5-4ef3-bfc2-df5ee9e62ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:05.389450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8f7c602b-0da5-4ef3-bfc2-df5ee9e62ca7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:05.390657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8f7c602b-0da5-4ef3-bfc2-df5ee9e62ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:05.453783 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1119/2230] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:05.463852 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:05.465297 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:05.497678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:05.524197 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60304014-f325-49db-8cb9-63323d652b23 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:05.527997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010200aa-4b01-4913-afa6-68e148dfba35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:05.528657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010200aa-4b01-4913-afa6-68e148dfba35 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:05.528932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010200aa-4b01-4913-afa6-68e148dfba35 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:05.529196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010200aa-4b01-4913-afa6-68e148dfba35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:05.538410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010200aa-4b01-4913-afa6-68e148dfba35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:05.539855 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010200aa-4b01-4913-afa6-68e148dfba35 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:05.549519 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1112/2231] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:05.825828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c602b-0da5-4ef3-bfc2-df5ee9e62ca7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:05.827287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8f7c602b-0da5-4ef3-bfc2-df5ee9e62ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:05.840774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c602b-0da5-4ef3-bfc2-df5ee9e62ca7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:05.845980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:05.846625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:05.846964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:05.847220 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] PATH_INFO: `/v3/projects/97d02f8dc3124b13a0a64a39348f3fbf` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:05.848702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d02f8dc3124b13a0a64a39348f3fbf)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:05.851270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:05.901305 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d02f8dc3124b13a0a64a39348f3fbf', 'request_id': 'req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.906709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d02f8dc3124b13a0a64a39348f3fbf', 'request_id': 'req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.912698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d02f8dc3124b13a0a64a39348f3fbf', 'request_id': 'req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.927103 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] The token cache is being invalidate because project 97d02f8dc3124b13a0a64a39348f3fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:05.927397 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.927996 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:05.929465 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1120/2232] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:05 2022] DELETE /identity/v3/projects/97d02f8dc3124b13a0a64a39348f3fbf => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:05.933408 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010200aa-4b01-4913-afa6-68e148dfba35 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:05.943405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010200aa-4b01-4913-afa6-68e148dfba35 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:06.002251 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:06.002812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.003092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.003365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] PATH_INFO: `/v3/users/0a8495b01cbf46d8833ecf6d5563e4ec` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.004478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a8495b01cbf46d8833ecf6d5563e4ec)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:06.006657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:06.051005 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a8495b01cbf46d8833ecf6d5563e4ec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.051765 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a8495b01cbf46d8833ecf6d5563e4ec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.058900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a8495b01cbf46d8833ecf6d5563e4ec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.066918 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a8495b01cbf46d8833ecf6d5563e4ec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.078324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1d25188b-71fb-4eea-a77f-c5676f97e758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a8495b01cbf46d8833ecf6d5563e4ec'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.084156 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1113/2233] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:05 2022] DELETE /identity/v3/users/0a8495b01cbf46d8833ecf6d5563e4ec => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:06.358282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:06.360374 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:06.378547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b756a-a101-43aa-af09-6d6dd7f2a4c9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:06.442342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:06.443009 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.443287 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.443584 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] PATH_INFO: `/v3/projects/7fa76decffc847abbcdab96a42e92e88` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.444721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fa76decffc847abbcdab96a42e92e88)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:06.446628 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:06.467051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:06.493503 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fa76decffc847abbcdab96a42e92e88', 'request_id': 'req-2f14ca5e-db16-4012-9da5-26160640c0db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.499163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fa76decffc847abbcdab96a42e92e88', 'request_id': 'req-2f14ca5e-db16-4012-9da5-26160640c0db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.505643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fa76decffc847abbcdab96a42e92e88', 'request_id': 'req-2f14ca5e-db16-4012-9da5-26160640c0db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.520625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] The token cache is being invalidate because project 7fa76decffc847abbcdab96a42e92e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:06.520911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.521551 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2f14ca5e-db16-4012-9da5-26160640c0db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:06.523064 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1121/2234] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:06 2022] DELETE /identity/v3/projects/7fa76decffc847abbcdab96a42e92e88 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:06.528121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac43e40-fa2b-4c31-9e9a-f32c57c56f7e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.528702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac43e40-fa2b-4c31-9e9a-f32c57c56f7e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.528800 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac43e40-fa2b-4c31-9e9a-f32c57c56f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.549319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4ac43e40-fa2b-4c31-9e9a-f32c57c56f7e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:06.550449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4ac43e40-fa2b-4c31-9e9a-f32c57c56f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:06.598792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:06.616102 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1114/2235] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:06.625228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c010b-a5fc-418b-8c03-96d563af32b2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.625510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c010b-a5fc-418b-8c03-96d563af32b2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.625808 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c010b-a5fc-418b-8c03-96d563af32b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.645758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-518c010b-a5fc-418b-8c03-96d563af32b2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:06.647037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-518c010b-a5fc-418b-8c03-96d563af32b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:06.656551 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1122/2236] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:06.661193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac43e40-fa2b-4c31-9e9a-f32c57c56f7e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:06.671289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac43e40-fa2b-4c31-9e9a-f32c57c56f7e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:06.677360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77a33d-364d-4520-8122-5666d789992b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:06.678192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77a33d-364d-4520-8122-5666d789992b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.678684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77a33d-364d-4520-8122-5666d789992b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.679121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77a33d-364d-4520-8122-5666d789992b admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.680292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77a33d-364d-4520-8122-5666d789992b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:06.681125 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77a33d-364d-4520-8122-5666d789992b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:06.683458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77a33d-364d-4520-8122-5666d789992b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:06.697653 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1115/2237] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:06.702840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c010b-a5fc-418b-8c03-96d563af32b2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:06.712102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c010b-a5fc-418b-8c03-96d563af32b2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:06.717336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b7a4-4a9f-495f-84e1-8285c222fac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:06.717946 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b7a4-4a9f-495f-84e1-8285c222fac2 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.718241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b7a4-4a9f-495f-84e1-8285c222fac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.718511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b7a4-4a9f-495f-84e1-8285c222fac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.719163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76b7a4-4a9f-495f-84e1-8285c222fac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:06.721041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76b7a4-4a9f-495f-84e1-8285c222fac2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:06.749164 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1123/2238] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:06 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:06.754921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77a33d-364d-4520-8122-5666d789992b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:06.765178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77a33d-364d-4520-8122-5666d789992b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:06.824347 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a42861-7e28-4216-822e-b9d7f6cb91c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:06.824866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a42861-7e28-4216-822e-b9d7f6cb91c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.825140 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a42861-7e28-4216-822e-b9d7f6cb91c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.825458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a42861-7e28-4216-822e-b9d7f6cb91c5 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.826103 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a42861-7e28-4216-822e-b9d7f6cb91c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:06.828026 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a42861-7e28-4216-822e-b9d7f6cb91c5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:06.850307 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1116/2239] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:06 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:06.854555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b7a4-4a9f-495f-84e1-8285c222fac2 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:06.865260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b7a4-4a9f-495f-84e1-8285c222fac2 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:06.870835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc082d-211c-413a-9c8a-b02038bb1479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:06.871432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc082d-211c-413a-9c8a-b02038bb1479 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.871709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc082d-211c-413a-9c8a-b02038bb1479 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.871976 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc082d-211c-413a-9c8a-b02038bb1479 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.872706 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc082d-211c-413a-9c8a-b02038bb1479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:06.874308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc082d-211c-413a-9c8a-b02038bb1479 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:06.884516 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1124/2240] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:06.888410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a42861-7e28-4216-822e-b9d7f6cb91c5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:06.898685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a42861-7e28-4216-822e-b9d7f6cb91c5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:06.904679 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3800980-4f75-40de-b751-005b1c40b5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:06.905259 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3800980-4f75-40de-b751-005b1c40b5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.905535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3800980-4f75-40de-b751-005b1c40b5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.905797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3800980-4f75-40de-b751-005b1c40b5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.906427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3800980-4f75-40de-b751-005b1c40b5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:06.908064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3800980-4f75-40de-b751-005b1c40b5a2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:06.919168 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1117/2241] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:06.923871 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc082d-211c-413a-9c8a-b02038bb1479 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:06.933555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc082d-211c-413a-9c8a-b02038bb1479 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:06.938192 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c6b67-a7f1-46a6-8b0d-4164246bb7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:06.938858 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c6b67-a7f1-46a6-8b0d-4164246bb7aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.939153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c6b67-a7f1-46a6-8b0d-4164246bb7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.939398 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c6b67-a7f1-46a6-8b0d-4164246bb7aa admin admin] PATH_INFO: `/v3/projects/79c123267baf43bfbdd71c51cefc925a/users/e69921812f1046e89ac04275e9cc8ee7/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:06.953635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c6b67-a7f1-46a6-8b0d-4164246bb7aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c123267baf43bfbdd71c51cefc925a, user_id=e69921812f1046e89ac04275e9cc8ee7, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:06.956292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c6b67-a7f1-46a6-8b0d-4164246bb7aa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:06.971486 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1125/2242] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:06 2022] PUT /identity/v3/projects/79c123267baf43bfbdd71c51cefc925a/users/e69921812f1046e89ac04275e9cc8ee7/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:06.981837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e936ef3-cfba-43cf-9ab3-253f20202dc1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:06.982120 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e936ef3-cfba-43cf-9ab3-253f20202dc1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:06.982431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e936ef3-cfba-43cf-9ab3-253f20202dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:07.011135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9e936ef3-cfba-43cf-9ab3-253f20202dc1 None None] MFA Rules not processed for user `e69921812f1046e89ac04275e9cc8ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:07.012364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9e936ef3-cfba-43cf-9ab3-253f20202dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:07.078250 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1118/2243] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:07.082832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c6b67-a7f1-46a6-8b0d-4164246bb7aa admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:07.144556 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c6b67-a7f1-46a6-8b0d-4164246bb7aa admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:07.149789 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f28bc-81dc-4820-a037-c047eb6f9faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:07.150387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f28bc-81dc-4820-a037-c047eb6f9faa admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:07.150665 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f28bc-81dc-4820-a037-c047eb6f9faa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:07.150964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f28bc-81dc-4820-a037-c047eb6f9faa admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:07.151592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f28bc-81dc-4820-a037-c047eb6f9faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:07.153514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f28bc-81dc-4820-a037-c047eb6f9faa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:07.182170 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1126/2244] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:07 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:07.186291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e936ef3-cfba-43cf-9ab3-253f20202dc1 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:07.196993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e936ef3-cfba-43cf-9ab3-253f20202dc1 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:07.240646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:07.255234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b9adb-2992-4b3f-bc38-2af89335397d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:07.255865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b9adb-2992-4b3f-bc38-2af89335397d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:07.256141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b9adb-2992-4b3f-bc38-2af89335397d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:07.256410 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b9adb-2992-4b3f-bc38-2af89335397d admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:07.257041 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b9adb-2992-4b3f-bc38-2af89335397d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:07.258979 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b9adb-2992-4b3f-bc38-2af89335397d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:07.281847 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1119/2245] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:07 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:07.286160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f28bc-81dc-4820-a037-c047eb6f9faa admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:07.296832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f28bc-81dc-4820-a037-c047eb6f9faa admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:07.302059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb43ed-b56f-45c6-8e4c-593229b5ccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:07.302651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb43ed-b56f-45c6-8e4c-593229b5ccd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:07.302957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb43ed-b56f-45c6-8e4c-593229b5ccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:07.303226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb43ed-b56f-45c6-8e4c-593229b5ccd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:07.303861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb43ed-b56f-45c6-8e4c-593229b5ccd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:07.305455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb43ed-b56f-45c6-8e4c-593229b5ccd4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:07.315940 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1127/2246] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:56:07.320007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b9adb-2992-4b3f-bc38-2af89335397d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:07.330812 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b9adb-2992-4b3f-bc38-2af89335397d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:07.335498 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec83ad-c346-42e5-b208-32d923a2c298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:07.336156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec83ad-c346-42e5-b208-32d923a2c298 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:07.336428 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec83ad-c346-42e5-b208-32d923a2c298 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:07.336677 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec83ad-c346-42e5-b208-32d923a2c298 admin admin] PATH_INFO: `/v3/projects/65f2e9c0e11547bf9b767ef16f8777eb/users/dad4747212324217bb346fac6cfa8ae5/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:07.351893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec83ad-c346-42e5-b208-32d923a2c298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f2e9c0e11547bf9b767ef16f8777eb, user_id=dad4747212324217bb346fac6cfa8ae5, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:07.354520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec83ad-c346-42e5-b208-32d923a2c298 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:07.367259 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1120/2247] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:07 2022] PUT /identity/v3/projects/65f2e9c0e11547bf9b767ef16f8777eb/users/dad4747212324217bb346fac6cfa8ae5/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:07.371178 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb43ed-b56f-45c6-8e4c-593229b5ccd4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:07.381143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb43ed-b56f-45c6-8e4c-593229b5ccd4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:07.393617 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:07.440737 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde14b2-dbd5-49c6-858c-cc2f9f5df931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:07.441353 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde14b2-dbd5-49c6-858c-cc2f9f5df931 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:07.441632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde14b2-dbd5-49c6-858c-cc2f9f5df931 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:07.441905 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde14b2-dbd5-49c6-858c-cc2f9f5df931 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:07.442535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde14b2-dbd5-49c6-858c-cc2f9f5df931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:07.444180 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde14b2-dbd5-49c6-858c-cc2f9f5df931 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:07.454353 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1128/2248] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:07.458604 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec83ad-c346-42e5-b208-32d923a2c298 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:07.468958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec83ad-c346-42e5-b208-32d923a2c298 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:07.473696 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e92c31-6e4a-4b9a-bdc4-d8e5f1ae25fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:07.474333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e92c31-6e4a-4b9a-bdc4-d8e5f1ae25fb admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:07.474605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e92c31-6e4a-4b9a-bdc4-d8e5f1ae25fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:07.474907 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e92c31-6e4a-4b9a-bdc4-d8e5f1ae25fb admin admin] PATH_INFO: `/v3/projects/65f2e9c0e11547bf9b767ef16f8777eb/users/dad4747212324217bb346fac6cfa8ae5/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:07.476838 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e92c31-6e4a-4b9a-bdc4-d8e5f1ae25fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f2e9c0e11547bf9b767ef16f8777eb, user_id=dad4747212324217bb346fac6cfa8ae5, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:07.479492 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e92c31-6e4a-4b9a-bdc4-d8e5f1ae25fb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:07.489878 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1121/2249] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:07 2022] PUT /identity/v3/projects/65f2e9c0e11547bf9b767ef16f8777eb/users/dad4747212324217bb346fac6cfa8ae5/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:07.500562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c770f-d3ed-4a63-aaa9-4e160de9b9ff None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:07.500954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c770f-d3ed-4a63-aaa9-4e160de9b9ff None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:07.501399 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c770f-d3ed-4a63-aaa9-4e160de9b9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:07.539821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-189c770f-d3ed-4a63-aaa9-4e160de9b9ff None None] MFA Rules not processed for user `dad4747212324217bb346fac6cfa8ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:07.541027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-189c770f-d3ed-4a63-aaa9-4e160de9b9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:07.611666 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1129/2250] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:07.622899 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e92c31-6e4a-4b9a-bdc4-d8e5f1ae25fb admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:07.625160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d4e92c31-6e4a-4b9a-bdc4-d8e5f1ae25fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:07.686383 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e92c31-6e4a-4b9a-bdc4-d8e5f1ae25fb admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:07.690445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2696f-a915-4ccc-b76d-b35c8693a240 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:07.691071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2696f-a915-4ccc-b76d-b35c8693a240 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:07.691343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2696f-a915-4ccc-b76d-b35c8693a240 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:07.691608 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2696f-a915-4ccc-b76d-b35c8693a240 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:07.700874 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2696f-a915-4ccc-b76d-b35c8693a240 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:07.702265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2696f-a915-4ccc-b76d-b35c8693a240 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:07.711811 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1122/2251] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:08.347478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c770f-d3ed-4a63-aaa9-4e160de9b9ff None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:08.409078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c770f-d3ed-4a63-aaa9-4e160de9b9ff None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:08.413654 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:08.414279 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:08.414557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:08.414862 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] PATH_INFO: `/v3/users/e69921812f1046e89ac04275e9cc8ee7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:08.416046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69921812f1046e89ac04275e9cc8ee7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:08.417950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:08.459636 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:08.465507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69921812f1046e89ac04275e9cc8ee7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:08.472260 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69921812f1046e89ac04275e9cc8ee7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:08.472901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69921812f1046e89ac04275e9cc8ee7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:08.480362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69921812f1046e89ac04275e9cc8ee7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:08.492491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69921812f1046e89ac04275e9cc8ee7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:08.498584 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1130/2252] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:08 2022] DELETE /identity/v3/users/e69921812f1046e89ac04275e9cc8ee7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:08.503213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec1d90-76be-4af4-8bed-86d05c940a59 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:08.503497 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec1d90-76be-4af4-8bed-86d05c940a59 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:08.503762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec1d90-76be-4af4-8bed-86d05c940a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:08.522865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-93ec1d90-76be-4af4-8bed-86d05c940a59 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:08.524089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-93ec1d90-76be-4af4-8bed-86d05c940a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:08.585604 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1123/2253] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:56:08.595742 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:08.597557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:08.658037 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777b561-1f77-4ca9-829e-c58125137507 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:08.662032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb357e-24f1-4dbf-9ab6-ee2134ad57b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:08.662558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb357e-24f1-4dbf-9ab6-ee2134ad57b1 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:08.662857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb357e-24f1-4dbf-9ab6-ee2134ad57b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:08.663129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb357e-24f1-4dbf-9ab6-ee2134ad57b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:08.672620 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb357e-24f1-4dbf-9ab6-ee2134ad57b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:08.674350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb357e-24f1-4dbf-9ab6-ee2134ad57b1 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:08.684044 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1131/2254] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:08.986017 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec1d90-76be-4af4-8bed-86d05c940a59 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:08.987476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-93ec1d90-76be-4af4-8bed-86d05c940a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:08.999807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec1d90-76be-4af4-8bed-86d05c940a59 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:09.004531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:09.005163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:09.005437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:09.005709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] PATH_INFO: `/v3/projects/79c123267baf43bfbdd71c51cefc925a` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:09.006999 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c123267baf43bfbdd71c51cefc925a)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:09.008838 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:09.048291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:09.059060 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c123267baf43bfbdd71c51cefc925a', 'request_id': 'req-ff40af82-5d73-49c6-8a0d-f15ebf295d10'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.064651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c123267baf43bfbdd71c51cefc925a', 'request_id': 'req-ff40af82-5d73-49c6-8a0d-f15ebf295d10'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.070837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c123267baf43bfbdd71c51cefc925a', 'request_id': 'req-ff40af82-5d73-49c6-8a0d-f15ebf295d10'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.083847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] The token cache is being invalidate because project 79c123267baf43bfbdd71c51cefc925a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:09.084130 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.084737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.086178 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1124/2255] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:08 2022] DELETE /identity/v3/projects/79c123267baf43bfbdd71c51cefc925a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:09.090268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb357e-24f1-4dbf-9ab6-ee2134ad57b1 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:09.100226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb357e-24f1-4dbf-9ab6-ee2134ad57b1 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:09.155317 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:09.155943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:09.156215 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:09.156480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] PATH_INFO: `/v3/users/dad4747212324217bb346fac6cfa8ae5` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:09.157562 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad4747212324217bb346fac6cfa8ae5)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:09.159539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:09.208657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad4747212324217bb346fac6cfa8ae5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.217267 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad4747212324217bb346fac6cfa8ae5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.217996 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad4747212324217bb346fac6cfa8ae5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.225830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad4747212324217bb346fac6cfa8ae5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.226962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:09.239783 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cab004a2-881e-46ce-b9c5-45bb2454a928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad4747212324217bb346fac6cfa8ae5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.245557 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1132/2256] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:09 2022] DELETE /identity/v3/users/dad4747212324217bb346fac6cfa8ae5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:09.528256 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:09.529710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:09.544349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40af82-5d73-49c6-8a0d-f15ebf295d10 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:09.601242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:09.601872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:09.602146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:09.602412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] PATH_INFO: `/v3/projects/65f2e9c0e11547bf9b767ef16f8777eb` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:09.603665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f2e9c0e11547bf9b767ef16f8777eb)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:09.605563 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:09.654357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f2e9c0e11547bf9b767ef16f8777eb', 'request_id': 'req-72b6c41b-3488-4354-8752-088096d81e13'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.659898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f2e9c0e11547bf9b767ef16f8777eb', 'request_id': 'req-72b6c41b-3488-4354-8752-088096d81e13'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.666178 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f2e9c0e11547bf9b767ef16f8777eb', 'request_id': 'req-72b6c41b-3488-4354-8752-088096d81e13'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.680357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] The token cache is being invalidate because project 65f2e9c0e11547bf9b767ef16f8777eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:09.680639 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.681260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-72b6c41b-3488-4354-8752-088096d81e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:09.682759 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1125/2257] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:09 2022] DELETE /identity/v3/projects/65f2e9c0e11547bf9b767ef16f8777eb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:56:09.687688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e67c-90b5-404e-b232-2a0010cd0ca5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:09.688186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e67c-90b5-404e-b232-2a0010cd0ca5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:09.688303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e67c-90b5-404e-b232-2a0010cd0ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:09.708574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9053e67c-90b5-404e-b232-2a0010cd0ca5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:09.709758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9053e67c-90b5-404e-b232-2a0010cd0ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:09.784254 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1133/2258] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:09.854606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a06a9d-ecfe-410d-b2fc-78a1ebc81f5e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:09.854891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a06a9d-ecfe-410d-b2fc-78a1ebc81f5e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:09.855207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a06a9d-ecfe-410d-b2fc-78a1ebc81f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:09.876412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-63a06a9d-ecfe-410d-b2fc-78a1ebc81f5e None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:09.877618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-63a06a9d-ecfe-410d-b2fc-78a1ebc81f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:09.886373 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1126/2259] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:09.890985 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053e67c-90b5-404e-b232-2a0010cd0ca5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:09.901335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053e67c-90b5-404e-b232-2a0010cd0ca5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:09.907056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0406eff-ec2b-49f9-bffa-27428deffae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:09.907658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0406eff-ec2b-49f9-bffa-27428deffae9 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:09.907932 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0406eff-ec2b-49f9-bffa-27428deffae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:09.908200 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0406eff-ec2b-49f9-bffa-27428deffae9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:09.908851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0406eff-ec2b-49f9-bffa-27428deffae9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:09.909144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0406eff-ec2b-49f9-bffa-27428deffae9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:09.910745 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0406eff-ec2b-49f9-bffa-27428deffae9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:09.922774 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1134/2260] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:09.927335 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a06a9d-ecfe-410d-b2fc-78a1ebc81f5e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:09.936728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a06a9d-ecfe-410d-b2fc-78a1ebc81f5e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:09.942012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf53bf-ca41-40da-abe3-136f11cf4f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:09.942601 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf53bf-ca41-40da-abe3-136f11cf4f1b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:09.942921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf53bf-ca41-40da-abe3-136f11cf4f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:09.943188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf53bf-ca41-40da-abe3-136f11cf4f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:09.943806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf53bf-ca41-40da-abe3-136f11cf4f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:09.945702 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf53bf-ca41-40da-abe3-136f11cf4f1b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:09.977540 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1127/2261] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:09 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:09.983379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0406eff-ec2b-49f9-bffa-27428deffae9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:09.993369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0406eff-ec2b-49f9-bffa-27428deffae9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.050293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fe42a-a67f-49dd-8667-b6077500d16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.050861 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fe42a-a67f-49dd-8667-b6077500d16a admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.051161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fe42a-a67f-49dd-8667-b6077500d16a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.051428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fe42a-a67f-49dd-8667-b6077500d16a admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.052050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fe42a-a67f-49dd-8667-b6077500d16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.053965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fe42a-a67f-49dd-8667-b6077500d16a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.076083 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1135/2262] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:09 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:10.080240 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf53bf-ca41-40da-abe3-136f11cf4f1b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.090987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf53bf-ca41-40da-abe3-136f11cf4f1b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.096843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5f1d-e477-481b-8050-dd4429903f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.097459 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5f1d-e477-481b-8050-dd4429903f1d admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.097730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5f1d-e477-481b-8050-dd4429903f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.097998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff5f1d-e477-481b-8050-dd4429903f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.098670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff5f1d-e477-481b-8050-dd4429903f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.100270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff5f1d-e477-481b-8050-dd4429903f1d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.110207 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1128/2263] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:10.114142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fe42a-a67f-49dd-8667-b6077500d16a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.124383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fe42a-a67f-49dd-8667-b6077500d16a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.130023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5784d91-97fb-4184-b058-0aa212f555f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.130866 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5784d91-97fb-4184-b058-0aa212f555f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.131157 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5784d91-97fb-4184-b058-0aa212f555f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.131449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5784d91-97fb-4184-b058-0aa212f555f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.132074 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5784d91-97fb-4184-b058-0aa212f555f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.133696 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5784d91-97fb-4184-b058-0aa212f555f4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.144847 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1136/2264] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:10.149564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5f1d-e477-481b-8050-dd4429903f1d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.159472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff5f1d-e477-481b-8050-dd4429903f1d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.164348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653a86c-35d6-4b48-aa75-df7aedd92e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.164982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653a86c-35d6-4b48-aa75-df7aedd92e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.165254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653a86c-35d6-4b48-aa75-df7aedd92e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.165525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653a86c-35d6-4b48-aa75-df7aedd92e38 admin admin] PATH_INFO: `/v3/projects/d5a3837a017845f2b2e9ce5fe2bb37e5/users/ca6d819903b4434ebe2884ad091af180/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.178023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:10.181668 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653a86c-35d6-4b48-aa75-df7aedd92e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a3837a017845f2b2e9ce5fe2bb37e5, user_id=ca6d819903b4434ebe2884ad091af180, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.184550 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653a86c-35d6-4b48-aa75-df7aedd92e38 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.198843 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1129/2265] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:10 2022] PUT /identity/v3/projects/d5a3837a017845f2b2e9ce5fe2bb37e5/users/ca6d819903b4434ebe2884ad091af180/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:10.209049 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b76f42-2387-42e1-b4b3-aea67964cc33 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.209336 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b76f42-2387-42e1-b4b3-aea67964cc33 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.209610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b76f42-2387-42e1-b4b3-aea67964cc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.237853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e3b76f42-2387-42e1-b4b3-aea67964cc33 None None] MFA Rules not processed for user `ca6d819903b4434ebe2884ad091af180`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:10.239116 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e3b76f42-2387-42e1-b4b3-aea67964cc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:10.272116 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:10.307805 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1137/2266] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:10.312564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653a86c-35d6-4b48-aa75-df7aedd92e38 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.385523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653a86c-35d6-4b48-aa75-df7aedd92e38 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.392359 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ab65-1d13-44f7-9417-daeb64e8381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.392949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ab65-1d13-44f7-9417-daeb64e8381b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.393226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ab65-1d13-44f7-9417-daeb64e8381b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.393493 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ab65-1d13-44f7-9417-daeb64e8381b admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.394124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ab65-1d13-44f7-9417-daeb64e8381b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.396042 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ab65-1d13-44f7-9417-daeb64e8381b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.424270 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1130/2267] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:10 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:10.428653 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b76f42-2387-42e1-b4b3-aea67964cc33 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.439104 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b76f42-2387-42e1-b4b3-aea67964cc33 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.494928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0638a-dc70-421e-a531-1d0031f60931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.495555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0638a-dc70-421e-a531-1d0031f60931 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.495815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0638a-dc70-421e-a531-1d0031f60931 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.496083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0638a-dc70-421e-a531-1d0031f60931 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.496704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0638a-dc70-421e-a531-1d0031f60931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.498651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0638a-dc70-421e-a531-1d0031f60931 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.520971 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1138/2268] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:10 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:10.526324 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ab65-1d13-44f7-9417-daeb64e8381b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.536641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ab65-1d13-44f7-9417-daeb64e8381b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.541276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46cc05-14ee-4c2d-9f6e-28e4590671b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.541865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46cc05-14ee-4c2d-9f6e-28e4590671b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.542143 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46cc05-14ee-4c2d-9f6e-28e4590671b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.542414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46cc05-14ee-4c2d-9f6e-28e4590671b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.543083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46cc05-14ee-4c2d-9f6e-28e4590671b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.544666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46cc05-14ee-4c2d-9f6e-28e4590671b5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.555707 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1131/2269] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:10.559934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0638a-dc70-421e-a531-1d0031f60931 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.571456 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0638a-dc70-421e-a531-1d0031f60931 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.579962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80089304-a0f3-4f71-8a5e-ae54743d43dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.580812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089304-a0f3-4f71-8a5e-ae54743d43dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.581180 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089304-a0f3-4f71-8a5e-ae54743d43dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.581553 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089304-a0f3-4f71-8a5e-ae54743d43dc admin admin] PATH_INFO: `/v3/projects/10673fc8bf184317959db5595e749d45/users/7d71854570c14fffb8ac4102407dcf8e/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.600008 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80089304-a0f3-4f71-8a5e-ae54743d43dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=10673fc8bf184317959db5595e749d45, user_id=7d71854570c14fffb8ac4102407dcf8e, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.602671 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80089304-a0f3-4f71-8a5e-ae54743d43dc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.616210 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1139/2270] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:10 2022] PUT /identity/v3/projects/10673fc8bf184317959db5595e749d45/users/7d71854570c14fffb8ac4102407dcf8e/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:56:10.620353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46cc05-14ee-4c2d-9f6e-28e4590671b5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.631020 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46cc05-14ee-4c2d-9f6e-28e4590671b5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.689770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24da1f-106c-4be2-995e-35e4b72af91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.690367 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24da1f-106c-4be2-995e-35e4b72af91b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.690645 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24da1f-106c-4be2-995e-35e4b72af91b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.691038 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24da1f-106c-4be2-995e-35e4b72af91b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.691671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24da1f-106c-4be2-995e-35e4b72af91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.693258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24da1f-106c-4be2-995e-35e4b72af91b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.703147 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1132/2271] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:10.707494 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80089304-a0f3-4f71-8a5e-ae54743d43dc admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.718520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80089304-a0f3-4f71-8a5e-ae54743d43dc admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.724120 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cd036-2645-43a2-8b28-f4f6ec0f49e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.724775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cd036-2645-43a2-8b28-f4f6ec0f49e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.725051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cd036-2645-43a2-8b28-f4f6ec0f49e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.725322 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cd036-2645-43a2-8b28-f4f6ec0f49e6 admin admin] PATH_INFO: `/v3/projects/10673fc8bf184317959db5595e749d45/users/7d71854570c14fffb8ac4102407dcf8e/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.727558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cd036-2645-43a2-8b28-f4f6ec0f49e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10673fc8bf184317959db5595e749d45, user_id=7d71854570c14fffb8ac4102407dcf8e, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.730184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cd036-2645-43a2-8b28-f4f6ec0f49e6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.744725 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1140/2272] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:10 2022] PUT /identity/v3/projects/10673fc8bf184317959db5595e749d45/users/7d71854570c14fffb8ac4102407dcf8e/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:10.757802 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2f1d7-0214-4a5a-ae7b-9d226e40a694 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.758091 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2f1d7-0214-4a5a-ae7b-9d226e40a694 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.758357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2f1d7-0214-4a5a-ae7b-9d226e40a694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.796683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-aae2f1d7-0214-4a5a-ae7b-9d226e40a694 None None] MFA Rules not processed for user `7d71854570c14fffb8ac4102407dcf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:10.797896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-aae2f1d7-0214-4a5a-ae7b-9d226e40a694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:10.847214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:10.871106 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1133/2273] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:56:10.882481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cd036-2645-43a2-8b28-f4f6ec0f49e6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:10.884555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f01cd036-2645-43a2-8b28-f4f6ec0f49e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:10.942532 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cd036-2645-43a2-8b28-f4f6ec0f49e6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:10.947556 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a856f-4aaf-4bec-9e99-262e53f0d0be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:10.948201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a856f-4aaf-4bec-9e99-262e53f0d0be service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:10.948471 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a856f-4aaf-4bec-9e99-262e53f0d0be service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:10.948721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a856f-4aaf-4bec-9e99-262e53f0d0be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:10.959949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a856f-4aaf-4bec-9e99-262e53f0d0be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:10.961381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a856f-4aaf-4bec-9e99-262e53f0d0be service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:10.971097 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1141/2274] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:11.361631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2f1d7-0214-4a5a-ae7b-9d226e40a694 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:11.425477 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2f1d7-0214-4a5a-ae7b-9d226e40a694 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:11.431106 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:11.431726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:11.432002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:11.432265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] PATH_INFO: `/v3/users/ca6d819903b4434ebe2884ad091af180` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:11.433512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6d819903b4434ebe2884ad091af180)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:11.435511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:11.481983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6d819903b4434ebe2884ad091af180'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:11.482678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6d819903b4434ebe2884ad091af180'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:11.490145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6d819903b4434ebe2884ad091af180'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:11.498456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6d819903b4434ebe2884ad091af180'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:11.510184 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6d819903b4434ebe2884ad091af180'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:11.516021 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1134/2275] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:11 2022] DELETE /identity/v3/users/ca6d819903b4434ebe2884ad091af180 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:11.520628 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4897e-9d42-44ff-93f7-a2f4fa073ee8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:11.520912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4897e-9d42-44ff-93f7-a2f4fa073ee8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:11.521183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4897e-9d42-44ff-93f7-a2f4fa073ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:11.541687 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-00b4897e-9d42-44ff-93f7-a2f4fa073ee8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:11.542920 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-00b4897e-9d42-44ff-93f7-a2f4fa073ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:11.567875 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:11.599928 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1142/2276] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:11.609743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:11.611175 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:11.670526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28eded-b212-4b64-b471-f46ae82e850f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:11.675288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0043c-bfdd-4672-9569-f888baa29f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:11.675881 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0043c-bfdd-4672-9569-f888baa29f15 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:11.676157 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0043c-bfdd-4672-9569-f888baa29f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:11.676427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0043c-bfdd-4672-9569-f888baa29f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:11.685713 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0043c-bfdd-4672-9569-f888baa29f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:11.687176 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0043c-bfdd-4672-9569-f888baa29f15 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:11.696459 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1135/2277] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:11.984443 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4897e-9d42-44ff-93f7-a2f4fa073ee8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:11.986136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-00b4897e-9d42-44ff-93f7-a2f4fa073ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:12.003204 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4897e-9d42-44ff-93f7-a2f4fa073ee8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:12.009973 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:12.010522 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:12.010812 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:12.011210 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] PATH_INFO: `/v3/projects/d5a3837a017845f2b2e9ce5fe2bb37e5` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:12.012656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a3837a017845f2b2e9ce5fe2bb37e5)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:12.014809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:12.066541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a3837a017845f2b2e9ce5fe2bb37e5', 'request_id': 'req-0bdfc283-bcaa-448c-a3f7-a1514f45c079'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.072547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a3837a017845f2b2e9ce5fe2bb37e5', 'request_id': 'req-0bdfc283-bcaa-448c-a3f7-a1514f45c079'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.078920 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a3837a017845f2b2e9ce5fe2bb37e5', 'request_id': 'req-0bdfc283-bcaa-448c-a3f7-a1514f45c079'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.093476 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] The token cache is being invalidate because project d5a3837a017845f2b2e9ce5fe2bb37e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:12.093763 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.094381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.095947 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1143/2278] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:11 2022] DELETE /identity/v3/projects/d5a3837a017845f2b2e9ce5fe2bb37e5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:12.100022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0043c-bfdd-4672-9569-f888baa29f15 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:12.109463 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0043c-bfdd-4672-9569-f888baa29f15 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:12.113926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:12.167665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:12.168280 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:12.168555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:12.168824 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] PATH_INFO: `/v3/users/7d71854570c14fffb8ac4102407dcf8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:12.169993 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d71854570c14fffb8ac4102407dcf8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:12.171922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:12.219089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d71854570c14fffb8ac4102407dcf8e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.219715 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d71854570c14fffb8ac4102407dcf8e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.226613 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d71854570c14fffb8ac4102407dcf8e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.234596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d71854570c14fffb8ac4102407dcf8e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.246393 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2842b507-a551-4299-96cc-633b077f8098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d71854570c14fffb8ac4102407dcf8e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.252252 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1136/2279] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:12 2022] DELETE /identity/v3/users/7d71854570c14fffb8ac4102407dcf8e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:12.532947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:12.534557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:12.558460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfc283-bcaa-448c-a3f7-a1514f45c079 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:12.619768 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:12.620408 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:12.620684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:12.620949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] PATH_INFO: `/v3/projects/10673fc8bf184317959db5595e749d45` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:12.622249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] RBAC: Authorizing `identity:delete_project(project_id=10673fc8bf184317959db5595e749d45)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:12.624863 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:12.675802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10673fc8bf184317959db5595e749d45', 'request_id': 'req-4279cae5-223c-4bcc-961d-ea74fe87901f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.680290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:12.681722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10673fc8bf184317959db5595e749d45', 'request_id': 'req-4279cae5-223c-4bcc-961d-ea74fe87901f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.688547 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10673fc8bf184317959db5595e749d45', 'request_id': 'req-4279cae5-223c-4bcc-961d-ea74fe87901f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.702816 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] The token cache is being invalidate because project 10673fc8bf184317959db5595e749d45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:12.703249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.703884 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4279cae5-223c-4bcc-961d-ea74fe87901f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:12.705374 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1144/2280] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:12 2022] DELETE /identity/v3/projects/10673fc8bf184317959db5595e749d45 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:12.710460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd3046-7d59-4af8-92ac-17b02c23eba8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:12.710967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd3046-7d59-4af8-92ac-17b02c23eba8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:12.711202 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd3046-7d59-4af8-92ac-17b02c23eba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:12.731203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-33dd3046-7d59-4af8-92ac-17b02c23eba8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:12.732579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-33dd3046-7d59-4af8-92ac-17b02c23eba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:12.793590 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1137/2281] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:12.803119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1d1c9-2fda-4864-881b-b20e0d384b28 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:12.803515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1d1c9-2fda-4864-881b-b20e0d384b28 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:12.803852 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1d1c9-2fda-4864-881b-b20e0d384b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:12.824610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5ba1d1c9-2fda-4864-881b-b20e0d384b28 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:12.825892 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5ba1d1c9-2fda-4864-881b-b20e0d384b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:12.835659 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1145/2282] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:12.839957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd3046-7d59-4af8-92ac-17b02c23eba8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:12.850072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd3046-7d59-4af8-92ac-17b02c23eba8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:12.854757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c9f84-1cd5-429f-8d31-804668706bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:12.855475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c9f84-1cd5-429f-8d31-804668706bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:12.855757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c9f84-1cd5-429f-8d31-804668706bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:12.856032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c9f84-1cd5-429f-8d31-804668706bb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:12.856735 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c9f84-1cd5-429f-8d31-804668706bb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:12.857041 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c9f84-1cd5-429f-8d31-804668706bb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:12.858638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c9f84-1cd5-429f-8d31-804668706bb7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:12.870865 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1138/2283] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:12.875347 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1d1c9-2fda-4864-881b-b20e0d384b28 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:12.885173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1d1c9-2fda-4864-881b-b20e0d384b28 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:12.891069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38c9a2-0744-43d3-8386-1e3c16050fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:12.891606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38c9a2-0744-43d3-8386-1e3c16050fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:12.891910 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38c9a2-0744-43d3-8386-1e3c16050fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:12.892183 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38c9a2-0744-43d3-8386-1e3c16050fd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:12.892868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38c9a2-0744-43d3-8386-1e3c16050fd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:12.894798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38c9a2-0744-43d3-8386-1e3c16050fd3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:12.924848 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1146/2284] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:12 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:12.930301 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c9f84-1cd5-429f-8d31-804668706bb7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:12.943269 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c9f84-1cd5-429f-8d31-804668706bb7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:12.955835 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:13.007366 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41c295-d452-4acb-8c5e-0b8fb41009df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:13.007982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41c295-d452-4acb-8c5e-0b8fb41009df admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:13.008267 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41c295-d452-4acb-8c5e-0b8fb41009df admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:13.008583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41c295-d452-4acb-8c5e-0b8fb41009df admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:13.009190 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41c295-d452-4acb-8c5e-0b8fb41009df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:13.011150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41c295-d452-4acb-8c5e-0b8fb41009df admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:13.033099 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1139/2285] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:12 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:13.037344 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38c9a2-0744-43d3-8386-1e3c16050fd3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:13.048549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38c9a2-0744-43d3-8386-1e3c16050fd3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:13.053940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d5a7d-9c85-4ee2-ac8c-599f3c75ea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:13.054591 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d5a7d-9c85-4ee2-ac8c-599f3c75ea0b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:13.055077 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d5a7d-9c85-4ee2-ac8c-599f3c75ea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:13.055231 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d5a7d-9c85-4ee2-ac8c-599f3c75ea0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:13.055921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d5a7d-9c85-4ee2-ac8c-599f3c75ea0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:13.057756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d5a7d-9c85-4ee2-ac8c-599f3c75ea0b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:13.068084 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1147/2286] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:13.072106 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41c295-d452-4acb-8c5e-0b8fb41009df admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:13.082620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41c295-d452-4acb-8c5e-0b8fb41009df admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:13.088087 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1eeba-f96d-4215-a547-cbc962745fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:13.088675 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1eeba-f96d-4215-a547-cbc962745fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:13.088981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1eeba-f96d-4215-a547-cbc962745fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:13.089241 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1eeba-f96d-4215-a547-cbc962745fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:13.089879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1eeba-f96d-4215-a547-cbc962745fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:13.091646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1eeba-f96d-4215-a547-cbc962745fa3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:13.102388 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1140/2287] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:13.112040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d5a7d-9c85-4ee2-ac8c-599f3c75ea0b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:13.122472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d5a7d-9c85-4ee2-ac8c-599f3c75ea0b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:13.127354 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa4265-634b-4c6c-8520-eb40beea8739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:13.128020 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa4265-634b-4c6c-8520-eb40beea8739 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:13.128303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa4265-634b-4c6c-8520-eb40beea8739 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:13.128575 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa4265-634b-4c6c-8520-eb40beea8739 admin admin] PATH_INFO: `/v3/projects/6d2806e58f714af7a7f739a59d3dece1/users/a7c4228e92254a65ada3a0a777812103/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:13.143257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa4265-634b-4c6c-8520-eb40beea8739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2806e58f714af7a7f739a59d3dece1, user_id=a7c4228e92254a65ada3a0a777812103, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:13.146216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa4265-634b-4c6c-8520-eb40beea8739 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:13.160673 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1148/2288] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:13 2022] PUT /identity/v3/projects/6d2806e58f714af7a7f739a59d3dece1/users/a7c4228e92254a65ada3a0a777812103/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:13.165872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7f15-d99a-430b-abda-d0936102bc86 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:13.166299 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7f15-d99a-430b-abda-d0936102bc86 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:13.166582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7f15-d99a-430b-abda-d0936102bc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:13.187512 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bd5b7f15-d99a-430b-abda-d0936102bc86 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:13.188713 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bd5b7f15-d99a-430b-abda-d0936102bc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:13.246820 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1141/2289] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:13.257294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa4265-634b-4c6c-8520-eb40beea8739 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:13.259874 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d6fa4265-634b-4c6c-8520-eb40beea8739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:13.323065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa4265-634b-4c6c-8520-eb40beea8739 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:13.326870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021cc1e-de42-4768-9f91-b0352548a932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:13.327473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cc1e-de42-4768-9f91-b0352548a932 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:13.327751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cc1e-de42-4768-9f91-b0352548a932 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:13.328023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cc1e-de42-4768-9f91-b0352548a932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:13.337175 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021cc1e-de42-4768-9f91-b0352548a932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:13.338590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021cc1e-de42-4768-9f91-b0352548a932 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:13.347943 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1149/2290] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:17.661220 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2127b1-8f35-4697-b4fc-ebedd6d7d806 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:17.661785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2127b1-8f35-4697-b4fc-ebedd6d7d806 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:17.661870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2127b1-8f35-4697-b4fc-ebedd6d7d806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:17.691901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6f2127b1-8f35-4697-b4fc-ebedd6d7d806 None None] MFA Rules not processed for user `a7c4228e92254a65ada3a0a777812103`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:17.693194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6f2127b1-8f35-4697-b4fc-ebedd6d7d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:17.764796 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1142/2291] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:17.769727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021cc1e-de42-4768-9f91-b0352548a932 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:17.780238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021cc1e-de42-4768-9f91-b0352548a932 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:17.789384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d87be4-cfdf-487d-931c-5cce033652aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:17.790101 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d87be4-cfdf-487d-931c-5cce033652aa admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:17.790442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d87be4-cfdf-487d-931c-5cce033652aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:17.790811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d87be4-cfdf-487d-931c-5cce033652aa admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:17.791535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d87be4-cfdf-487d-931c-5cce033652aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:17.793629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d87be4-cfdf-487d-931c-5cce033652aa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:17.827081 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1150/2292] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:17 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:17.831090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2127b1-8f35-4697-b4fc-ebedd6d7d806 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:17.842326 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2127b1-8f35-4697-b4fc-ebedd6d7d806 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:17.870252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:17.907212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15fe94-2135-40f5-8855-ccf9231cfade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:17.907817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15fe94-2135-40f5-8855-ccf9231cfade admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:17.908090 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15fe94-2135-40f5-8855-ccf9231cfade admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:17.908355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15fe94-2135-40f5-8855-ccf9231cfade admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:17.908992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15fe94-2135-40f5-8855-ccf9231cfade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:17.910912 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15fe94-2135-40f5-8855-ccf9231cfade admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:17.933068 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1143/2293] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:17 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:17.937415 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d87be4-cfdf-487d-931c-5cce033652aa admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:17.948145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d87be4-cfdf-487d-931c-5cce033652aa admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:17.952764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61a77e-1cd8-423f-a518-414128a677e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:17.953399 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61a77e-1cd8-423f-a518-414128a677e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:17.953680 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61a77e-1cd8-423f-a518-414128a677e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:17.953941 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61a77e-1cd8-423f-a518-414128a677e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:17.954640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61a77e-1cd8-423f-a518-414128a677e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:17.956186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61a77e-1cd8-423f-a518-414128a677e6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:17.972579 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1151/2294] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:17.976823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15fe94-2135-40f5-8855-ccf9231cfade admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:17.987013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15fe94-2135-40f5-8855-ccf9231cfade admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:17.991770 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae3ecf-9094-41ae-a1b0-68dcf3180eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:17.992428 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae3ecf-9094-41ae-a1b0-68dcf3180eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:17.992703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae3ecf-9094-41ae-a1b0-68dcf3180eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:17.992974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae3ecf-9094-41ae-a1b0-68dcf3180eb3 admin admin] PATH_INFO: `/v3/projects/a49ac070323b4b42a925286738643569/users/8030cebfa4324a639e1d813e6aa092ce/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:18.009019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae3ecf-9094-41ae-a1b0-68dcf3180eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49ac070323b4b42a925286738643569, user_id=8030cebfa4324a639e1d813e6aa092ce, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:18.011844 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae3ecf-9094-41ae-a1b0-68dcf3180eb3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:18.026353 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1144/2295] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:17 2022] PUT /identity/v3/projects/a49ac070323b4b42a925286738643569/users/8030cebfa4324a639e1d813e6aa092ce/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:18.030481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61a77e-1cd8-423f-a518-414128a677e6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:18.040743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61a77e-1cd8-423f-a518-414128a677e6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:18.087038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:18.097958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56f9b1-236b-4642-aa9d-bb2160c3eaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:18.098631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56f9b1-236b-4642-aa9d-bb2160c3eaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:18.098943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56f9b1-236b-4642-aa9d-bb2160c3eaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:18.099225 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56f9b1-236b-4642-aa9d-bb2160c3eaa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:18.099876 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56f9b1-236b-4642-aa9d-bb2160c3eaa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:18.101614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56f9b1-236b-4642-aa9d-bb2160c3eaa2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:18.111773 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1152/2296] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:18.116028 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae3ecf-9094-41ae-a1b0-68dcf3180eb3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:18.126518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae3ecf-9094-41ae-a1b0-68dcf3180eb3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:18.131528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1c865-d7c8-4c8f-a2c2-c30499bf81ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:18.132175 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1c865-d7c8-4c8f-a2c2-c30499bf81ed admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:18.132454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1c865-d7c8-4c8f-a2c2-c30499bf81ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:18.132739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1c865-d7c8-4c8f-a2c2-c30499bf81ed admin admin] PATH_INFO: `/v3/projects/a49ac070323b4b42a925286738643569/users/8030cebfa4324a639e1d813e6aa092ce/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:18.134651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1c865-d7c8-4c8f-a2c2-c30499bf81ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49ac070323b4b42a925286738643569, user_id=8030cebfa4324a639e1d813e6aa092ce, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:18.137296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1c865-d7c8-4c8f-a2c2-c30499bf81ed admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:18.147497 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1145/2297] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:18 2022] PUT /identity/v3/projects/a49ac070323b4b42a925286738643569/users/8030cebfa4324a639e1d813e6aa092ce/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:23.644611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182846c-c6f0-4672-a4c8-359cb716922f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:23.645119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182846c-c6f0-4672-a4c8-359cb716922f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:23.645119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182846c-c6f0-4672-a4c8-359cb716922f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:23.676121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7182846c-c6f0-4672-a4c8-359cb716922f None None] MFA Rules not processed for user `8030cebfa4324a639e1d813e6aa092ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:23.677341 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7182846c-c6f0-4672-a4c8-359cb716922f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:23.745654 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1153/2298] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:23.759379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1c865-d7c8-4c8f-a2c2-c30499bf81ed admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:23.761233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f8e1c865-d7c8-4c8f-a2c2-c30499bf81ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:23.837596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1c865-d7c8-4c8f-a2c2-c30499bf81ed admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:23.845661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba737b74-85ca-4aa0-81e4-92bb74afa135 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:23.847208 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba737b74-85ca-4aa0-81e4-92bb74afa135 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:23.847488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba737b74-85ca-4aa0-81e4-92bb74afa135 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:23.847760 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba737b74-85ca-4aa0-81e4-92bb74afa135 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:23.910290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba737b74-85ca-4aa0-81e4-92bb74afa135 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:23.912454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba737b74-85ca-4aa0-81e4-92bb74afa135 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:23.921618 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1146/2299] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:24.979315 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a070800-f2a6-4c91-b8fc-964a2cc31e8b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:24.979870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a070800-f2a6-4c91-b8fc-964a2cc31e8b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:24.979870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a070800-f2a6-4c91-b8fc-964a2cc31e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:24.999249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0a070800-f2a6-4c91-b8fc-964a2cc31e8b None None] MFA Rules not processed for user `a7c4228e92254a65ada3a0a777812103`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:25.000468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0a070800-f2a6-4c91-b8fc-964a2cc31e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:25.007970 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1154/2300] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:26.650704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba737b74-85ca-4aa0-81e4-92bb74afa135 service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:26.652164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ba737b74-85ca-4aa0-81e4-92bb74afa135 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:26.667731 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:26.729483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba737b74-85ca-4aa0-81e4-92bb74afa135 service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:26.734145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4cc7f-986f-4a77-a5b1-8a982cbbd5fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:26.734746 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4cc7f-986f-4a77-a5b1-8a982cbbd5fa service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:26.735040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4cc7f-986f-4a77-a5b1-8a982cbbd5fa service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:26.735307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4cc7f-986f-4a77-a5b1-8a982cbbd5fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:26.737225 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-41b4cc7f-986f-4a77-a5b1-8a982cbbd5fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:26.805218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4cc7f-986f-4a77-a5b1-8a982cbbd5fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:26.806553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4cc7f-986f-4a77-a5b1-8a982cbbd5fa service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:26.816173 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1147/2301] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:30.326858 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a070800-f2a6-4c91-b8fc-964a2cc31e8b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:30.328207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0a070800-f2a6-4c91-b8fc-964a2cc31e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:30.342061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a070800-f2a6-4c91-b8fc-964a2cc31e8b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:30.346891 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:30.347535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:30.347811 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:30.348072 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] PATH_INFO: `/v3/users/712e5cf4cc004918817ab1802b9c69d6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:30.349233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] RBAC: Authorizing `identity:delete_user(user_id=712e5cf4cc004918817ab1802b9c69d6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:30.351208 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:30.397197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712e5cf4cc004918817ab1802b9c69d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.404065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712e5cf4cc004918817ab1802b9c69d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.404788 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712e5cf4cc004918817ab1802b9c69d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.413956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712e5cf4cc004918817ab1802b9c69d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.434062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4bfeb526-2ae6-41ec-bd4a-9bee305d203d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712e5cf4cc004918817ab1802b9c69d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.440537 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1155/2302] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:30 2022] DELETE /identity/v3/users/712e5cf4cc004918817ab1802b9c69d6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:30.618853 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4cc7f-986f-4a77-a5b1-8a982cbbd5fa service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:30.630737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4cc7f-986f-4a77-a5b1-8a982cbbd5fa service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:30.692900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:30.693525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:30.693803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:30.694071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] PATH_INFO: `/v3/projects/02be1b30677f400788b35b03938e44e5` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:30.695207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] RBAC: Authorizing `identity:delete_project(project_id=02be1b30677f400788b35b03938e44e5)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:30.697089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:30.752495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02be1b30677f400788b35b03938e44e5', 'request_id': 'req-8927ea04-c4ee-4225-99a9-699f71b3d55a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.759920 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02be1b30677f400788b35b03938e44e5', 'request_id': 'req-8927ea04-c4ee-4225-99a9-699f71b3d55a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.766667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02be1b30677f400788b35b03938e44e5', 'request_id': 'req-8927ea04-c4ee-4225-99a9-699f71b3d55a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.769405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:30.782960 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] The token cache is being invalidate because project 02be1b30677f400788b35b03938e44e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:30.783416 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.784080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8927ea04-c4ee-4225-99a9-699f71b3d55a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:30.785667 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1148/2303] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:30 2022] DELETE /identity/v3/projects/02be1b30677f400788b35b03938e44e5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:30.790603 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8472b-4eb4-4709-bb79-d7886afc1005 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:30.790927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8472b-4eb4-4709-bb79-d7886afc1005 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:30.791201 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8472b-4eb4-4709-bb79-d7886afc1005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:30.812733 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-efe8472b-4eb4-4709-bb79-d7886afc1005 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:30.814528 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-efe8472b-4eb4-4709-bb79-d7886afc1005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:30.849617 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:30.878547 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1156/2304] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:30.886430 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ca265-5546-463d-9db0-783e5477be14 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:30.886783 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ca265-5546-463d-9db0-783e5477be14 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:30.887059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ca265-5546-463d-9db0-783e5477be14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:30.909289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-985ca265-5546-463d-9db0-783e5477be14 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:30.910576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-985ca265-5546-463d-9db0-783e5477be14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:30.919371 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1149/2305] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:30.923984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8472b-4eb4-4709-bb79-d7886afc1005 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:30.934059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8472b-4eb4-4709-bb79-d7886afc1005 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:30.939075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712f361-1140-4ee9-a230-87f573265cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:30.939672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712f361-1140-4ee9-a230-87f573265cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:30.939950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712f361-1140-4ee9-a230-87f573265cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:30.940228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712f361-1140-4ee9-a230-87f573265cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:30.940850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712f361-1140-4ee9-a230-87f573265cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:30.941188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712f361-1140-4ee9-a230-87f573265cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:30.942797 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712f361-1140-4ee9-a230-87f573265cd4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:30.954490 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1157/2306] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:30.958963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ca265-5546-463d-9db0-783e5477be14 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:30.968443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ca265-5546-463d-9db0-783e5477be14 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:30.974919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d7cfc-9c69-4926-9b49-2ab8d4e43fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:30.975889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d7cfc-9c69-4926-9b49-2ab8d4e43fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:30.976175 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d7cfc-9c69-4926-9b49-2ab8d4e43fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:30.976468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d7cfc-9c69-4926-9b49-2ab8d4e43fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:30.977472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d7cfc-9c69-4926-9b49-2ab8d4e43fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:30.979893 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d7cfc-9c69-4926-9b49-2ab8d4e43fb6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.016174 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1150/2307] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:30 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:31.017907 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712f361-1140-4ee9-a230-87f573265cd4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.028244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712f361-1140-4ee9-a230-87f573265cd4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.095795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba653a4-b89b-4080-a31d-e0e60adcb4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.096525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba653a4-b89b-4080-a31d-e0e60adcb4bb admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.096870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba653a4-b89b-4080-a31d-e0e60adcb4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.097207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba653a4-b89b-4080-a31d-e0e60adcb4bb admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.097870 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba653a4-b89b-4080-a31d-e0e60adcb4bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.099905 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba653a4-b89b-4080-a31d-e0e60adcb4bb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.121808 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1158/2308] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:31 2022] POST /identity/v3/users => generated 281 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:31.125607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d7cfc-9c69-4926-9b49-2ab8d4e43fb6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.135895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d7cfc-9c69-4926-9b49-2ab8d4e43fb6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.140580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ed64f-c43f-4a6f-85e9-60830c8925f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.141204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ed64f-c43f-4a6f-85e9-60830c8925f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.141487 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ed64f-c43f-4a6f-85e9-60830c8925f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.141757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ed64f-c43f-4a6f-85e9-60830c8925f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.142393 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ed64f-c43f-4a6f-85e9-60830c8925f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.144021 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ed64f-c43f-4a6f-85e9-60830c8925f5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.155452 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1151/2309] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:31.159625 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba653a4-b89b-4080-a31d-e0e60adcb4bb admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.170114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba653a4-b89b-4080-a31d-e0e60adcb4bb admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.175390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf8a99-ee66-4dc8-97fc-25c8e11ec09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.176010 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8a99-ee66-4dc8-97fc-25c8e11ec09d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.176289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8a99-ee66-4dc8-97fc-25c8e11ec09d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.176714 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8a99-ee66-4dc8-97fc-25c8e11ec09d admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.177498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf8a99-ee66-4dc8-97fc-25c8e11ec09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.179173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf8a99-ee66-4dc8-97fc-25c8e11ec09d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.194757 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1159/2310] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:31.196227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ed64f-c43f-4a6f-85e9-60830c8925f5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.206392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ed64f-c43f-4a6f-85e9-60830c8925f5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.210901 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43f271-778d-405c-bec8-cfec566e482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.211543 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43f271-778d-405c-bec8-cfec566e482e admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.211817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43f271-778d-405c-bec8-cfec566e482e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.212159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43f271-778d-405c-bec8-cfec566e482e admin admin] PATH_INFO: `/v3/projects/eae55a0ac47c4777a756ab94bd8a297a/users/1798a10e03f9406ebfe4cfe2d25891c7/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.227234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43f271-778d-405c-bec8-cfec566e482e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae55a0ac47c4777a756ab94bd8a297a, user_id=1798a10e03f9406ebfe4cfe2d25891c7, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.229957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43f271-778d-405c-bec8-cfec566e482e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.244198 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1152/2311] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:31 2022] PUT /identity/v3/projects/eae55a0ac47c4777a756ab94bd8a297a/users/1798a10e03f9406ebfe4cfe2d25891c7/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:31.254696 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26c316-a6a3-417f-bbb3-2317c4481c79 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.254944 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26c316-a6a3-417f-bbb3-2317c4481c79 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.255244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26c316-a6a3-417f-bbb3-2317c4481c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.294989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bb26c316-a6a3-417f-bbb3-2317c4481c79 None None] MFA Rules not processed for user `1798a10e03f9406ebfe4cfe2d25891c7`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:31.296244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bb26c316-a6a3-417f-bbb3-2317c4481c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:31.374508 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1160/2312] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:31.379143 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43f271-778d-405c-bec8-cfec566e482e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.441462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43f271-778d-405c-bec8-cfec566e482e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.446309 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70ca65-f1d6-4393-8eea-2355214419b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.446936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70ca65-f1d6-4393-8eea-2355214419b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.447214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70ca65-f1d6-4393-8eea-2355214419b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.447483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70ca65-f1d6-4393-8eea-2355214419b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.448161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70ca65-f1d6-4393-8eea-2355214419b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.450069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70ca65-f1d6-4393-8eea-2355214419b0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.480032 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1153/2313] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:31 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:31.484397 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26c316-a6a3-417f-bbb3-2317c4481c79 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.495245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26c316-a6a3-417f-bbb3-2317c4481c79 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.529329 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:31.559544 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46621247-27f1-4f2b-b61d-f5510851a191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.560326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46621247-27f1-4f2b-b61d-f5510851a191 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.560544 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46621247-27f1-4f2b-b61d-f5510851a191 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.560849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46621247-27f1-4f2b-b61d-f5510851a191 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.561649 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46621247-27f1-4f2b-b61d-f5510851a191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.563631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46621247-27f1-4f2b-b61d-f5510851a191 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.586174 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1161/2314] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:31 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:31.590263 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70ca65-f1d6-4393-8eea-2355214419b0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.602119 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70ca65-f1d6-4393-8eea-2355214419b0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.609008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f4e3f-207c-49d8-a0e3-abb7ecd9c88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.609636 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f4e3f-207c-49d8-a0e3-abb7ecd9c88a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.609946 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f4e3f-207c-49d8-a0e3-abb7ecd9c88a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.610221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f4e3f-207c-49d8-a0e3-abb7ecd9c88a admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.610919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f4e3f-207c-49d8-a0e3-abb7ecd9c88a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.612614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f4e3f-207c-49d8-a0e3-abb7ecd9c88a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.624836 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1154/2315] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:31.629382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46621247-27f1-4f2b-b61d-f5510851a191 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.640003 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46621247-27f1-4f2b-b61d-f5510851a191 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.645291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c2f24-727e-40c8-bb01-9cb29dba861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.645938 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c2f24-727e-40c8-bb01-9cb29dba861c admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.646224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c2f24-727e-40c8-bb01-9cb29dba861c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.646496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c2f24-727e-40c8-bb01-9cb29dba861c admin admin] PATH_INFO: `/v3/projects/5d198e2e50e94b7e95072742bbd28996/users/24ddfedf45af4ed6876a75036b4cacab/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.661428 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c2f24-727e-40c8-bb01-9cb29dba861c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d198e2e50e94b7e95072742bbd28996, user_id=24ddfedf45af4ed6876a75036b4cacab, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.664129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c2f24-727e-40c8-bb01-9cb29dba861c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.679228 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1162/2316] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:31 2022] PUT /identity/v3/projects/5d198e2e50e94b7e95072742bbd28996/users/24ddfedf45af4ed6876a75036b4cacab/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:31.683321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f4e3f-207c-49d8-a0e3-abb7ecd9c88a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.694034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f4e3f-207c-49d8-a0e3-abb7ecd9c88a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.752037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a6306-cc63-433a-a124-48f0a085e127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.752641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6306-cc63-433a-a124-48f0a085e127 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.752922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6306-cc63-433a-a124-48f0a085e127 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.753194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6306-cc63-433a-a124-48f0a085e127 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.753913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a6306-cc63-433a-a124-48f0a085e127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.755546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a6306-cc63-433a-a124-48f0a085e127 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.760813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:31.766403 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1155/2317] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:31.770762 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c2f24-727e-40c8-bb01-9cb29dba861c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.782401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c2f24-727e-40c8-bb01-9cb29dba861c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:31.789437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce33ad-111b-4107-83b9-06a5e7478b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:31.790441 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce33ad-111b-4107-83b9-06a5e7478b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.791057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce33ad-111b-4107-83b9-06a5e7478b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.791442 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce33ad-111b-4107-83b9-06a5e7478b84 admin admin] PATH_INFO: `/v3/projects/5d198e2e50e94b7e95072742bbd28996/users/24ddfedf45af4ed6876a75036b4cacab/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.793677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce33ad-111b-4107-83b9-06a5e7478b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d198e2e50e94b7e95072742bbd28996, user_id=24ddfedf45af4ed6876a75036b4cacab, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:31.796688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce33ad-111b-4107-83b9-06a5e7478b84 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:31.810546 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1163/2318] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:31 2022] PUT /identity/v3/projects/5d198e2e50e94b7e95072742bbd28996/users/24ddfedf45af4ed6876a75036b4cacab/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:31.822343 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea319658-7968-4814-adf4-deba740b316d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:31.822628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea319658-7968-4814-adf4-deba740b316d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:31.822936 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea319658-7968-4814-adf4-deba740b316d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:31.851691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ea319658-7968-4814-adf4-deba740b316d None None] MFA Rules not processed for user `24ddfedf45af4ed6876a75036b4cacab`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:31.852897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ea319658-7968-4814-adf4-deba740b316d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:31.924998 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1156/2319] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:31.936565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce33ad-111b-4107-83b9-06a5e7478b84 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:31.938487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3bce33ad-111b-4107-83b9-06a5e7478b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:32.001435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce33ad-111b-4107-83b9-06a5e7478b84 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:32.005894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f8979-6b4d-4103-80bc-7cb7c3a6304f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:32.006587 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8979-6b4d-4103-80bc-7cb7c3a6304f service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:32.006900 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8979-6b4d-4103-80bc-7cb7c3a6304f service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:32.007160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f8979-6b4d-4103-80bc-7cb7c3a6304f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:32.017226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f8979-6b4d-4103-80bc-7cb7c3a6304f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:32.018695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f8979-6b4d-4103-80bc-7cb7c3a6304f service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:32.028566 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1164/2320] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:32.465373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea319658-7968-4814-adf4-deba740b316d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:32.529369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea319658-7968-4814-adf4-deba740b316d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:32.539680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:32.540257 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:32.540531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:32.540800 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] PATH_INFO: `/v3/users/1798a10e03f9406ebfe4cfe2d25891c7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:32.541976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1798a10e03f9406ebfe4cfe2d25891c7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:32.543968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:32.597692 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1798a10e03f9406ebfe4cfe2d25891c7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:32.598372 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1798a10e03f9406ebfe4cfe2d25891c7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:32.605839 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1798a10e03f9406ebfe4cfe2d25891c7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:32.614863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1798a10e03f9406ebfe4cfe2d25891c7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:32.628127 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1798a10e03f9406ebfe4cfe2d25891c7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:32.635501 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1157/2321] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:32 2022] DELETE /identity/v3/users/1798a10e03f9406ebfe4cfe2d25891c7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:32.640086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c0c06-6698-4a59-977c-1588930fd9d8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:32.640614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c0c06-6698-4a59-977c-1588930fd9d8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:32.640614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c0c06-6698-4a59-977c-1588930fd9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:32.660112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1b9c0c06-6698-4a59-977c-1588930fd9d8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:32.661358 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1b9c0c06-6698-4a59-977c-1588930fd9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:32.719824 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1165/2322] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:32.730458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:32.732159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:32.739380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:32.794480 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1e14-76b0-4370-bc06-261bff23098f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:32.799062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be431076-0534-4816-83a6-09ab9b14741a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:32.799674 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be431076-0534-4816-83a6-09ab9b14741a service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:32.799957 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be431076-0534-4816-83a6-09ab9b14741a service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:32.800235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be431076-0534-4816-83a6-09ab9b14741a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:32.810978 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be431076-0534-4816-83a6-09ab9b14741a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:32.812439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be431076-0534-4816-83a6-09ab9b14741a service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:32.822375 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1158/2323] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:56:33.112420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c0c06-6698-4a59-977c-1588930fd9d8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:33.114049 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1b9c0c06-6698-4a59-977c-1588930fd9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:33.126626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c0c06-6698-4a59-977c-1588930fd9d8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:33.131490 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:33.132269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:33.132615 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:33.132956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] PATH_INFO: `/v3/projects/eae55a0ac47c4777a756ab94bd8a297a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:33.134148 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae55a0ac47c4777a756ab94bd8a297a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:33.136195 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:33.172736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:33.197585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae55a0ac47c4777a756ab94bd8a297a', 'request_id': 'req-1d6a5ac7-029b-4b06-a7fd-628b0759163d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.202911 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae55a0ac47c4777a756ab94bd8a297a', 'request_id': 'req-1d6a5ac7-029b-4b06-a7fd-628b0759163d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.209046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae55a0ac47c4777a756ab94bd8a297a', 'request_id': 'req-1d6a5ac7-029b-4b06-a7fd-628b0759163d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.222382 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] The token cache is being invalidate because project eae55a0ac47c4777a756ab94bd8a297a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:33.222698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.223346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.225152 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1166/2324] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:33 2022] DELETE /identity/v3/projects/eae55a0ac47c4777a756ab94bd8a297a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:33.228894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be431076-0534-4816-83a6-09ab9b14741a service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:33.238596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be431076-0534-4816-83a6-09ab9b14741a service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:33.309526 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:33.310187 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:33.310464 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:33.310758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] PATH_INFO: `/v3/users/24ddfedf45af4ed6876a75036b4cacab` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:33.311898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ddfedf45af4ed6876a75036b4cacab)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:33.313796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:33.364122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ddfedf45af4ed6876a75036b4cacab'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.364767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ddfedf45af4ed6876a75036b4cacab'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.373833 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ddfedf45af4ed6876a75036b4cacab'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.386247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ddfedf45af4ed6876a75036b4cacab'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.398777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-11a55fff-012f-4427-8788-a425c49f7577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ddfedf45af4ed6876a75036b4cacab'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.404737 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1159/2325] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:33 2022] DELETE /identity/v3/users/24ddfedf45af4ed6876a75036b4cacab => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:33.696557 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:33.698053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:33.713144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a5ac7-029b-4b06-a7fd-628b0759163d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:33.771033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:33.771676 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:33.771954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:33.772228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] PATH_INFO: `/v3/projects/5d198e2e50e94b7e95072742bbd28996` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:33.773415 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d198e2e50e94b7e95072742bbd28996)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:33.775369 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:33.823756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d198e2e50e94b7e95072742bbd28996', 'request_id': 'req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.829022 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d198e2e50e94b7e95072742bbd28996', 'request_id': 'req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.834934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d198e2e50e94b7e95072742bbd28996', 'request_id': 'req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.848118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] The token cache is being invalidate because project 5d198e2e50e94b7e95072742bbd28996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:33.848397 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.849007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f34b8cfb-fe9d-4f00-bf3f-914d8283f646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:33.850486 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1167/2326] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:33 2022] DELETE /identity/v3/projects/5d198e2e50e94b7e95072742bbd28996 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:33.855369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d086b1e-c4b3-49e8-847e-0bb4df7172a5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:33.855880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d086b1e-c4b3-49e8-847e-0bb4df7172a5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:33.855977 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d086b1e-c4b3-49e8-847e-0bb4df7172a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:33.878250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5d086b1e-c4b3-49e8-847e-0bb4df7172a5 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:33.879496 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5d086b1e-c4b3-49e8-847e-0bb4df7172a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:33.902692 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:33.948831 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1160/2327] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:33.961869 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827aaa8-6f76-4d98-93e5-3b103f5e6abf None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:33.962121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827aaa8-6f76-4d98-93e5-3b103f5e6abf None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:33.962440 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827aaa8-6f76-4d98-93e5-3b103f5e6abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:33.987901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d827aaa8-6f76-4d98-93e5-3b103f5e6abf None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:33.989153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d827aaa8-6f76-4d98-93e5-3b103f5e6abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:33.998830 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1168/2328] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:34.003734 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d086b1e-c4b3-49e8-847e-0bb4df7172a5 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.014063 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d086b1e-c4b3-49e8-847e-0bb4df7172a5 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.019340 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cc305-cf5f-43ea-8195-8b38e73d13a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.019966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cc305-cf5f-43ea-8195-8b38e73d13a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.020249 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cc305-cf5f-43ea-8195-8b38e73d13a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.020531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cc305-cf5f-43ea-8195-8b38e73d13a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.021234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cc305-cf5f-43ea-8195-8b38e73d13a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:34.021536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cc305-cf5f-43ea-8195-8b38e73d13a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.023250 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cc305-cf5f-43ea-8195-8b38e73d13a8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.036337 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1161/2329] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:34.040675 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827aaa8-6f76-4d98-93e5-3b103f5e6abf None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.050945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827aaa8-6f76-4d98-93e5-3b103f5e6abf None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.062753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085842b3-e0d6-45bd-bee0-adde5c3827ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.062753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085842b3-e0d6-45bd-bee0-adde5c3827ea admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.062753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085842b3-e0d6-45bd-bee0-adde5c3827ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.062753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085842b3-e0d6-45bd-bee0-adde5c3827ea admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.063452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085842b3-e0d6-45bd-bee0-adde5c3827ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.063452 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085842b3-e0d6-45bd-bee0-adde5c3827ea admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.101076 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1169/2330] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:34 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:34.105450 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cc305-cf5f-43ea-8195-8b38e73d13a8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.116034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cc305-cf5f-43ea-8195-8b38e73d13a8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.179903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925276d-847a-4d6b-b4ae-d98b5e997e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.180546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925276d-847a-4d6b-b4ae-d98b5e997e59 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.180825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925276d-847a-4d6b-b4ae-d98b5e997e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.181121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925276d-847a-4d6b-b4ae-d98b5e997e59 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.181733 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925276d-847a-4d6b-b4ae-d98b5e997e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.183738 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925276d-847a-4d6b-b4ae-d98b5e997e59 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.207348 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1162/2331] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:34 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:34.211584 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085842b3-e0d6-45bd-bee0-adde5c3827ea admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.221907 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085842b3-e0d6-45bd-bee0-adde5c3827ea admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.226792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed350b-06b9-4372-8166-923b0a7ab7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.227385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed350b-06b9-4372-8166-923b0a7ab7fa admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.227664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed350b-06b9-4372-8166-923b0a7ab7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.227936 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed350b-06b9-4372-8166-923b0a7ab7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.228574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed350b-06b9-4372-8166-923b0a7ab7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.230163 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed350b-06b9-4372-8166-923b0a7ab7fa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.239988 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1170/2332] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:34.243950 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925276d-847a-4d6b-b4ae-d98b5e997e59 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.254245 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925276d-847a-4d6b-b4ae-d98b5e997e59 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.260081 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a623-1aa6-4d94-ac5b-4c80ff064ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.260700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a623-1aa6-4d94-ac5b-4c80ff064ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.260978 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a623-1aa6-4d94-ac5b-4c80ff064ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.261465 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a623-1aa6-4d94-ac5b-4c80ff064ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.262232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a623-1aa6-4d94-ac5b-4c80ff064ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.264100 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a623-1aa6-4d94-ac5b-4c80ff064ef8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.275770 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1163/2333] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:56:34.280483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed350b-06b9-4372-8166-923b0a7ab7fa admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.285840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:34.292401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed350b-06b9-4372-8166-923b0a7ab7fa admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.299068 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fe416-c97b-4b3c-9ea9-95721df656eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.299866 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fe416-c97b-4b3c-9ea9-95721df656eb admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.300196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fe416-c97b-4b3c-9ea9-95721df656eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.300694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fe416-c97b-4b3c-9ea9-95721df656eb admin admin] PATH_INFO: `/v3/projects/94fcb8615a604cb5a51aad6a45b2f115/users/9adbc5de732843d1bcf5a283766fbc6a/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.330744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fe416-c97b-4b3c-9ea9-95721df656eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fcb8615a604cb5a51aad6a45b2f115, user_id=9adbc5de732843d1bcf5a283766fbc6a, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.334708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fe416-c97b-4b3c-9ea9-95721df656eb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.355444 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1171/2334] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:34 2022] PUT /identity/v3/projects/94fcb8615a604cb5a51aad6a45b2f115/users/9adbc5de732843d1bcf5a283766fbc6a/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:34.366149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b99a7-039e-431a-82c0-3d4ab870b1c0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.366412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b99a7-039e-431a-82c0-3d4ab870b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.366696 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b99a7-039e-431a-82c0-3d4ab870b1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.395809 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6c1b99a7-039e-431a-82c0-3d4ab870b1c0 None None] MFA Rules not processed for user `9adbc5de732843d1bcf5a283766fbc6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:34.397018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6c1b99a7-039e-431a-82c0-3d4ab870b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:34.475748 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1164/2335] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:34.480502 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fe416-c97b-4b3c-9ea9-95721df656eb admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.552614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fe416-c97b-4b3c-9ea9-95721df656eb admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.557690 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4e6c-80f8-417a-a8b1-db58b32ff9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.558265 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4e6c-80f8-417a-a8b1-db58b32ff9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.558542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4e6c-80f8-417a-a8b1-db58b32ff9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.558844 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4e6c-80f8-417a-a8b1-db58b32ff9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.559483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a4e6c-80f8-417a-a8b1-db58b32ff9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.561444 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a4e6c-80f8-417a-a8b1-db58b32ff9d4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.589577 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1172/2336] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:34 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:34.593757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b99a7-039e-431a-82c0-3d4ab870b1c0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.604924 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b99a7-039e-431a-82c0-3d4ab870b1c0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.623633 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:34.665766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4b1c7-e720-46a2-9213-189aa77b4127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.666373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b1c7-e720-46a2-9213-189aa77b4127 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.666655 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b1c7-e720-46a2-9213-189aa77b4127 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.666968 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4b1c7-e720-46a2-9213-189aa77b4127 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.667601 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4b1c7-e720-46a2-9213-189aa77b4127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.669527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4b1c7-e720-46a2-9213-189aa77b4127 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.695411 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1165/2337] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:34 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:34.696575 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4e6c-80f8-417a-a8b1-db58b32ff9d4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.707078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4e6c-80f8-417a-a8b1-db58b32ff9d4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.711908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd178a0-a6cc-4bdc-8772-6cca0b5e3d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.712501 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd178a0-a6cc-4bdc-8772-6cca0b5e3d82 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.712781 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd178a0-a6cc-4bdc-8772-6cca0b5e3d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.713057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd178a0-a6cc-4bdc-8772-6cca0b5e3d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.713700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd178a0-a6cc-4bdc-8772-6cca0b5e3d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.715335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd178a0-a6cc-4bdc-8772-6cca0b5e3d82 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.726194 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1173/2338] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:34.730454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4b1c7-e720-46a2-9213-189aa77b4127 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.741152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4b1c7-e720-46a2-9213-189aa77b4127 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.746182 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d10ae-193b-4aa0-bc67-65243f4b0da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.746856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d10ae-193b-4aa0-bc67-65243f4b0da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.747135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d10ae-193b-4aa0-bc67-65243f4b0da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.747579 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d10ae-193b-4aa0-bc67-65243f4b0da9 admin admin] PATH_INFO: `/v3/projects/7df2bf4a8cde4d16bc5bcf35f46c8371/users/eb574d93a25a409aa8e1f30a8f1ab87e/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.763826 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d10ae-193b-4aa0-bc67-65243f4b0da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df2bf4a8cde4d16bc5bcf35f46c8371, user_id=eb574d93a25a409aa8e1f30a8f1ab87e, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.766435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d10ae-193b-4aa0-bc67-65243f4b0da9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.781199 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1166/2339] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:34 2022] PUT /identity/v3/projects/7df2bf4a8cde4d16bc5bcf35f46c8371/users/eb574d93a25a409aa8e1f30a8f1ab87e/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:34.785418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd178a0-a6cc-4bdc-8772-6cca0b5e3d82 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.795601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd178a0-a6cc-4bdc-8772-6cca0b5e3d82 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.855898 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d789b-35a6-438c-be2e-2ae10e53730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.856507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d789b-35a6-438c-be2e-2ae10e53730f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.856778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d789b-35a6-438c-be2e-2ae10e53730f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.857044 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d789b-35a6-438c-be2e-2ae10e53730f admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.857684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d789b-35a6-438c-be2e-2ae10e53730f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.859392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d789b-35a6-438c-be2e-2ae10e53730f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.869140 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1174/2340] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:34.873499 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d10ae-193b-4aa0-bc67-65243f4b0da9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:34.884348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d10ae-193b-4aa0-bc67-65243f4b0da9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:34.889772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1c295-d3fe-4a81-97a6-9af06d15c029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:34.890407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1c295-d3fe-4a81-97a6-9af06d15c029 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.890684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1c295-d3fe-4a81-97a6-9af06d15c029 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.891001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1c295-d3fe-4a81-97a6-9af06d15c029 admin admin] PATH_INFO: `/v3/projects/7df2bf4a8cde4d16bc5bcf35f46c8371/users/eb574d93a25a409aa8e1f30a8f1ab87e/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.892976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1c295-d3fe-4a81-97a6-9af06d15c029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df2bf4a8cde4d16bc5bcf35f46c8371, user_id=eb574d93a25a409aa8e1f30a8f1ab87e, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:34.895611 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1c295-d3fe-4a81-97a6-9af06d15c029 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:34.905871 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1167/2341] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:34 2022] PUT /identity/v3/projects/7df2bf4a8cde4d16bc5bcf35f46c8371/users/eb574d93a25a409aa8e1f30a8f1ab87e/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:34.917147 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35a252-73cb-488c-9ce8-bd94b71963d8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:34.917440 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35a252-73cb-488c-9ce8-bd94b71963d8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:34.917809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35a252-73cb-488c-9ce8-bd94b71963d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:34.947425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6e35a252-73cb-488c-9ce8-bd94b71963d8 None None] MFA Rules not processed for user `eb574d93a25a409aa8e1f30a8f1ab87e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:34.948627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6e35a252-73cb-488c-9ce8-bd94b71963d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:34.991020 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:35.023668 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1175/2342] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:35.034725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1c295-d3fe-4a81-97a6-9af06d15c029 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:35.036399 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bcb1c295-d3fe-4a81-97a6-9af06d15c029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:35.098437 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1c295-d3fe-4a81-97a6-9af06d15c029 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:35.102736 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2d8e8-4e78-4f8c-903d-c63b4b2f651b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:35.103352 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2d8e8-4e78-4f8c-903d-c63b4b2f651b service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:35.103618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2d8e8-4e78-4f8c-903d-c63b4b2f651b service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:35.103919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2d8e8-4e78-4f8c-903d-c63b4b2f651b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:35.113551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2d8e8-4e78-4f8c-903d-c63b4b2f651b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:35.114996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2d8e8-4e78-4f8c-903d-c63b4b2f651b service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:35.124480 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1168/2343] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:36.547466 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35a252-73cb-488c-9ce8-bd94b71963d8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:36.613690 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35a252-73cb-488c-9ce8-bd94b71963d8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:36.618940 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:36.619606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:36.619883 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:36.620146 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] PATH_INFO: `/v3/users/9adbc5de732843d1bcf5a283766fbc6a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:36.621356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9adbc5de732843d1bcf5a283766fbc6a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:36.623349 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:36.672612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9adbc5de732843d1bcf5a283766fbc6a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:36.679984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9adbc5de732843d1bcf5a283766fbc6a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:36.680672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9adbc5de732843d1bcf5a283766fbc6a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:36.689887 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9adbc5de732843d1bcf5a283766fbc6a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:36.702046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9adbc5de732843d1bcf5a283766fbc6a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:36.707993 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1176/2344] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:36 2022] DELETE /identity/v3/users/9adbc5de732843d1bcf5a283766fbc6a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:36.712530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4bc77-140b-4954-abc0-583cc403eca6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:36.713013 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4bc77-140b-4954-abc0-583cc403eca6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:36.713112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4bc77-140b-4954-abc0-583cc403eca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:36.732524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-eda4bc77-140b-4954-abc0-583cc403eca6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:36.733694 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-eda4bc77-140b-4954-abc0-583cc403eca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:36.794028 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1169/2345] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:56:36.804956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:36.807381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:36.867771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3980f3-5aac-4ade-9485-a7c119a095d9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:36.872465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a0b90-35ae-4867-8bad-31731b467c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:36.873094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a0b90-35ae-4867-8bad-31731b467c45 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:36.873365 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a0b90-35ae-4867-8bad-31731b467c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:36.873631 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a0b90-35ae-4867-8bad-31731b467c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:36.881259 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:36.883914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a0b90-35ae-4867-8bad-31731b467c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:36.885484 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a0b90-35ae-4867-8bad-31731b467c45 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:36.897650 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1177/2346] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:37.176476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4bc77-140b-4954-abc0-583cc403eca6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:37.178049 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-eda4bc77-140b-4954-abc0-583cc403eca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:37.190942 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4bc77-140b-4954-abc0-583cc403eca6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:37.196262 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:37.196980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:37.197394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:37.197782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] PATH_INFO: `/v3/projects/94fcb8615a604cb5a51aad6a45b2f115` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:37.199055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=94fcb8615a604cb5a51aad6a45b2f115)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:37.201395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:37.213862 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:37.254302 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94fcb8615a604cb5a51aad6a45b2f115', 'request_id': 'req-e6847760-dfff-41f9-a0af-e88f1e99bc7a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.260074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94fcb8615a604cb5a51aad6a45b2f115', 'request_id': 'req-e6847760-dfff-41f9-a0af-e88f1e99bc7a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.266980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94fcb8615a604cb5a51aad6a45b2f115', 'request_id': 'req-e6847760-dfff-41f9-a0af-e88f1e99bc7a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.280670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] The token cache is being invalidate because project 94fcb8615a604cb5a51aad6a45b2f115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:37.281019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.281637 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.283129 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1170/2347] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:37 2022] DELETE /identity/v3/projects/94fcb8615a604cb5a51aad6a45b2f115 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:37.287495 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a0b90-35ae-4867-8bad-31731b467c45 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:37.297793 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a0b90-35ae-4867-8bad-31731b467c45 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:37.356007 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:37.356659 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:37.356978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:37.357249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] PATH_INFO: `/v3/users/eb574d93a25a409aa8e1f30a8f1ab87e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:37.358392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb574d93a25a409aa8e1f30a8f1ab87e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:37.360339 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:37.407493 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb574d93a25a409aa8e1f30a8f1ab87e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.414202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb574d93a25a409aa8e1f30a8f1ab87e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.414897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb574d93a25a409aa8e1f30a8f1ab87e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.423905 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb574d93a25a409aa8e1f30a8f1ab87e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.435535 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6c3b06b4-23b8-4e90-9e35-1c00fd736fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb574d93a25a409aa8e1f30a8f1ab87e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.441696 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1178/2348] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:37 2022] DELETE /identity/v3/users/eb574d93a25a409aa8e1f30a8f1ab87e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:56:37.722631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:37.724122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:37.740409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6847760-dfff-41f9-a0af-e88f1e99bc7a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:37.799531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:37.800183 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:37.800458 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:37.800726 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] PATH_INFO: `/v3/projects/7df2bf4a8cde4d16bc5bcf35f46c8371` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:37.801919 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df2bf4a8cde4d16bc5bcf35f46c8371)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:37.803854 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:37.852038 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df2bf4a8cde4d16bc5bcf35f46c8371', 'request_id': 'req-de19d937-61a1-4647-9738-d3b7f97de513'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.858107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df2bf4a8cde4d16bc5bcf35f46c8371', 'request_id': 'req-de19d937-61a1-4647-9738-d3b7f97de513'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.864507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df2bf4a8cde4d16bc5bcf35f46c8371', 'request_id': 'req-de19d937-61a1-4647-9738-d3b7f97de513'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.879011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] The token cache is being invalidate because project 7df2bf4a8cde4d16bc5bcf35f46c8371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:37.879270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.879967 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-de19d937-61a1-4647-9738-d3b7f97de513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:37.881474 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1171/2349] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:37 2022] DELETE /identity/v3/projects/7df2bf4a8cde4d16bc5bcf35f46c8371 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:37.887168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f23f4-688f-4d7f-b025-aab004c0bde3 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:37.887661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f23f4-688f-4d7f-b025-aab004c0bde3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:37.887778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f23f4-688f-4d7f-b025-aab004c0bde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:37.907612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-154f23f4-688f-4d7f-b025-aab004c0bde3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:37.908798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-154f23f4-688f-4d7f-b025-aab004c0bde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:37.975164 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1179/2350] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:37.985213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5c79a-8e7b-4a5b-8c93-ca19e2de93e7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:37.985500 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5c79a-8e7b-4a5b-8c93-ca19e2de93e7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:37.985767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5c79a-8e7b-4a5b-8c93-ca19e2de93e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.008640 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-aed5c79a-8e7b-4a5b-8c93-ca19e2de93e7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:38.009922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-aed5c79a-8e7b-4a5b-8c93-ca19e2de93e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:38.019920 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1172/2351] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:38.024349 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f23f4-688f-4d7f-b025-aab004c0bde3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.034564 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f23f4-688f-4d7f-b025-aab004c0bde3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.040086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06568f-8edc-4df1-8e85-d765cfacb545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.040694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06568f-8edc-4df1-8e85-d765cfacb545 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.040974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06568f-8edc-4df1-8e85-d765cfacb545 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.041244 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06568f-8edc-4df1-8e85-d765cfacb545 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.041901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06568f-8edc-4df1-8e85-d765cfacb545 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:38.042204 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06568f-8edc-4df1-8e85-d765cfacb545 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.043841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06568f-8edc-4df1-8e85-d765cfacb545 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.056188 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1180/2352] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:38.060485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5c79a-8e7b-4a5b-8c93-ca19e2de93e7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.070040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5c79a-8e7b-4a5b-8c93-ca19e2de93e7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.075194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f79f5e-4242-456e-8b08-cc9088dba476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.075837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f79f5e-4242-456e-8b08-cc9088dba476 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.076113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f79f5e-4242-456e-8b08-cc9088dba476 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.076402 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f79f5e-4242-456e-8b08-cc9088dba476 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.077027 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f79f5e-4242-456e-8b08-cc9088dba476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.078955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f79f5e-4242-456e-8b08-cc9088dba476 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.107963 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1173/2353] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:38 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:38.112219 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06568f-8edc-4df1-8e85-d765cfacb545 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.117576 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:38.125954 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06568f-8edc-4df1-8e85-d765cfacb545 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.186949 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee24ca-bc21-49e8-b34c-03e89759deb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.187558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee24ca-bc21-49e8-b34c-03e89759deb3 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.187850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee24ca-bc21-49e8-b34c-03e89759deb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.188121 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee24ca-bc21-49e8-b34c-03e89759deb3 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.188740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee24ca-bc21-49e8-b34c-03e89759deb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.190950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee24ca-bc21-49e8-b34c-03e89759deb3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.213013 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1181/2354] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:38 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:38.217109 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f79f5e-4242-456e-8b08-cc9088dba476 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.222589 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:38.229860 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f79f5e-4242-456e-8b08-cc9088dba476 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.235894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95849b31-9c76-48c2-b05b-a69950479ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.236483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95849b31-9c76-48c2-b05b-a69950479ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.236759 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95849b31-9c76-48c2-b05b-a69950479ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.237027 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95849b31-9c76-48c2-b05b-a69950479ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.237667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95849b31-9c76-48c2-b05b-a69950479ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.239279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95849b31-9c76-48c2-b05b-a69950479ff4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.249111 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1174/2355] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:38.253148 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee24ca-bc21-49e8-b34c-03e89759deb3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.263819 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee24ca-bc21-49e8-b34c-03e89759deb3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.269748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75f811-3481-487d-a7d7-7885148d110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.270254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75f811-3481-487d-a7d7-7885148d110d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.270534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75f811-3481-487d-a7d7-7885148d110d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.270824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75f811-3481-487d-a7d7-7885148d110d admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.271480 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75f811-3481-487d-a7d7-7885148d110d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.273079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75f811-3481-487d-a7d7-7885148d110d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.287800 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1182/2356] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:38.288683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95849b31-9c76-48c2-b05b-a69950479ff4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.299207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95849b31-9c76-48c2-b05b-a69950479ff4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.304501 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203910f8-7dcf-489a-8dbf-69d9bcf9c0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.305156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203910f8-7dcf-489a-8dbf-69d9bcf9c0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.305432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203910f8-7dcf-489a-8dbf-69d9bcf9c0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.305703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203910f8-7dcf-489a-8dbf-69d9bcf9c0e8 admin admin] PATH_INFO: `/v3/projects/6edb1f2a192247dd891b8e8063a2f8e2/users/3659030a0e7c465fb654c494e62fecf0/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.325782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203910f8-7dcf-489a-8dbf-69d9bcf9c0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edb1f2a192247dd891b8e8063a2f8e2, user_id=3659030a0e7c465fb654c494e62fecf0, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.328467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203910f8-7dcf-489a-8dbf-69d9bcf9c0e8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.342979 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1175/2357] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:38 2022] PUT /identity/v3/projects/6edb1f2a192247dd891b8e8063a2f8e2/users/3659030a0e7c465fb654c494e62fecf0/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:38.355238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095e13b-e46b-4ae4-835a-4a7ca5554a5e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.355541 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095e13b-e46b-4ae4-835a-4a7ca5554a5e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.355916 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095e13b-e46b-4ae4-835a-4a7ca5554a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.390526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e095e13b-e46b-4ae4-835a-4a7ca5554a5e None None] MFA Rules not processed for user `3659030a0e7c465fb654c494e62fecf0`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:38.391840 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e095e13b-e46b-4ae4-835a-4a7ca5554a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:38.455996 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1183/2358] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:38.460624 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203910f8-7dcf-489a-8dbf-69d9bcf9c0e8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.527042 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203910f8-7dcf-489a-8dbf-69d9bcf9c0e8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.532210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4116f6-7d1d-4dab-b8b9-63bc58728bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.532804 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4116f6-7d1d-4dab-b8b9-63bc58728bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.533080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4116f6-7d1d-4dab-b8b9-63bc58728bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.533351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4116f6-7d1d-4dab-b8b9-63bc58728bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.533983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4116f6-7d1d-4dab-b8b9-63bc58728bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.535913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4116f6-7d1d-4dab-b8b9-63bc58728bd2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.565177 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1176/2359] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:38 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:38.569458 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095e13b-e46b-4ae4-835a-4a7ca5554a5e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.580612 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095e13b-e46b-4ae4-835a-4a7ca5554a5e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.636250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf71b33-de3f-42b0-9101-621cd0d864fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.636878 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf71b33-de3f-42b0-9101-621cd0d864fb admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.637158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf71b33-de3f-42b0-9101-621cd0d864fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.637434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf71b33-de3f-42b0-9101-621cd0d864fb admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.638167 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf71b33-de3f-42b0-9101-621cd0d864fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.640203 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf71b33-de3f-42b0-9101-621cd0d864fb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.659948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:38.665526 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1184/2360] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:38 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:38.669813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4116f6-7d1d-4dab-b8b9-63bc58728bd2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.680283 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4116f6-7d1d-4dab-b8b9-63bc58728bd2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.685032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe00bcb-d3cb-4a35-b626-f8206adfa052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.685606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe00bcb-d3cb-4a35-b626-f8206adfa052 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.685882 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe00bcb-d3cb-4a35-b626-f8206adfa052 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.686171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe00bcb-d3cb-4a35-b626-f8206adfa052 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.686925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe00bcb-d3cb-4a35-b626-f8206adfa052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.688511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe00bcb-d3cb-4a35-b626-f8206adfa052 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.699140 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1177/2361] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:38.703448 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf71b33-de3f-42b0-9101-621cd0d864fb admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.713948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf71b33-de3f-42b0-9101-621cd0d864fb admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.718894 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428f224-fe8a-4e40-8728-55775f2d05d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.719561 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428f224-fe8a-4e40-8728-55775f2d05d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.719832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428f224-fe8a-4e40-8728-55775f2d05d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.720100 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428f224-fe8a-4e40-8728-55775f2d05d6 admin admin] PATH_INFO: `/v3/projects/663c40c676a245ea94b79698fb25fa6e/users/d97580922e364a8a9ff051c061b2f016/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.735477 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428f224-fe8a-4e40-8728-55775f2d05d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663c40c676a245ea94b79698fb25fa6e, user_id=d97580922e364a8a9ff051c061b2f016, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.738156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428f224-fe8a-4e40-8728-55775f2d05d6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.752867 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1185/2362] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:38 2022] PUT /identity/v3/projects/663c40c676a245ea94b79698fb25fa6e/users/d97580922e364a8a9ff051c061b2f016/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:38.758680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe00bcb-d3cb-4a35-b626-f8206adfa052 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.770295 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe00bcb-d3cb-4a35-b626-f8206adfa052 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.829674 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6921c-5b2f-4f47-a16a-582950719c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.830272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6921c-5b2f-4f47-a16a-582950719c02 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.830574 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6921c-5b2f-4f47-a16a-582950719c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.830876 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6921c-5b2f-4f47-a16a-582950719c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.831507 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6921c-5b2f-4f47-a16a-582950719c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.833061 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6921c-5b2f-4f47-a16a-582950719c02 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.842824 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1178/2363] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:38.847832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428f224-fe8a-4e40-8728-55775f2d05d6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:38.862206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428f224-fe8a-4e40-8728-55775f2d05d6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:38.868137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dd557-651a-48a4-b65b-bd7b5923a3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:38.868785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dd557-651a-48a4-b65b-bd7b5923a3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.869061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dd557-651a-48a4-b65b-bd7b5923a3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.869335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dd557-651a-48a4-b65b-bd7b5923a3c2 admin admin] PATH_INFO: `/v3/projects/663c40c676a245ea94b79698fb25fa6e/users/d97580922e364a8a9ff051c061b2f016/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.871464 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14dd557-651a-48a4-b65b-bd7b5923a3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663c40c676a245ea94b79698fb25fa6e, user_id=d97580922e364a8a9ff051c061b2f016, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:38.874171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14dd557-651a-48a4-b65b-bd7b5923a3c2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:38.885251 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1186/2364] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:38 2022] PUT /identity/v3/projects/663c40c676a245ea94b79698fb25fa6e/users/d97580922e364a8a9ff051c061b2f016/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:38.895303 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c47dd-ad35-4ba5-942d-f18db3e9a716 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:38.895588 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c47dd-ad35-4ba5-942d-f18db3e9a716 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:38.895880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c47dd-ad35-4ba5-942d-f18db3e9a716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:38.924641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-448c47dd-ad35-4ba5-942d-f18db3e9a716 None None] MFA Rules not processed for user `d97580922e364a8a9ff051c061b2f016`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:38.925854 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-448c47dd-ad35-4ba5-942d-f18db3e9a716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:38.953932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:38.998841 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1179/2365] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:39.012242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dd557-651a-48a4-b65b-bd7b5923a3c2 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:39.014051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c14dd557-651a-48a4-b65b-bd7b5923a3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:39.073172 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dd557-651a-48a4-b65b-bd7b5923a3c2 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:39.080061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b87144-a81f-4306-9dd8-c96fa02deba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:39.080936 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b87144-a81f-4306-9dd8-c96fa02deba7 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:39.081298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b87144-a81f-4306-9dd8-c96fa02deba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:39.081598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b87144-a81f-4306-9dd8-c96fa02deba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:39.095039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b87144-a81f-4306-9dd8-c96fa02deba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:39.096463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b87144-a81f-4306-9dd8-c96fa02deba7 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:39.106270 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1187/2366] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:39.136682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c47dd-ad35-4ba5-942d-f18db3e9a716 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:39.202474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c47dd-ad35-4ba5-942d-f18db3e9a716 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:39.207677 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:39.208276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:39.208548 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:39.208798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] PATH_INFO: `/v3/users/3659030a0e7c465fb654c494e62fecf0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:39.209926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3659030a0e7c465fb654c494e62fecf0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:39.211868 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:39.263019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3659030a0e7c465fb654c494e62fecf0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.263656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3659030a0e7c465fb654c494e62fecf0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.270661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3659030a0e7c465fb654c494e62fecf0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.280049 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3659030a0e7c465fb654c494e62fecf0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.292505 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3659030a0e7c465fb654c494e62fecf0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.298545 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1180/2367] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:39 2022] DELETE /identity/v3/users/3659030a0e7c465fb654c494e62fecf0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:39.303368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8671d-8e80-4e1e-868b-99e4b5ef5f06 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:39.303649 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8671d-8e80-4e1e-868b-99e4b5ef5f06 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:39.303918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8671d-8e80-4e1e-868b-99e4b5ef5f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:39.323013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-61d8671d-8e80-4e1e-868b-99e4b5ef5f06 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:39.324189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-61d8671d-8e80-4e1e-868b-99e4b5ef5f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:39.394306 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1188/2368] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:39.404496 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:39.406003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:39.456798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:39.475871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040c5d9-c4a1-47b9-b355-55008da80f57 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:39.482105 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b78fbe-1f88-4b5d-8c2a-35fd760cd4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:39.482749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b78fbe-1f88-4b5d-8c2a-35fd760cd4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:39.483043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b78fbe-1f88-4b5d-8c2a-35fd760cd4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:39.483307 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b78fbe-1f88-4b5d-8c2a-35fd760cd4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:39.493738 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b78fbe-1f88-4b5d-8c2a-35fd760cd4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:39.495193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b78fbe-1f88-4b5d-8c2a-35fd760cd4c8 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:39.504677 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1181/2369] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:39.768243 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8671d-8e80-4e1e-868b-99e4b5ef5f06 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:39.769848 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-61d8671d-8e80-4e1e-868b-99e4b5ef5f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:39.782240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8671d-8e80-4e1e-868b-99e4b5ef5f06 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:39.790038 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:39.790945 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:39.791223 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:39.791629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] PATH_INFO: `/v3/projects/6edb1f2a192247dd891b8e8063a2f8e2` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:39.792694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=6edb1f2a192247dd891b8e8063a2f8e2)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:39.794779 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:39.842058 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6edb1f2a192247dd891b8e8063a2f8e2', 'request_id': 'req-798e024a-cf9f-4a58-93f4-4b2628daf1ef'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.847736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6edb1f2a192247dd891b8e8063a2f8e2', 'request_id': 'req-798e024a-cf9f-4a58-93f4-4b2628daf1ef'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.849614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:39.854836 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6edb1f2a192247dd891b8e8063a2f8e2', 'request_id': 'req-798e024a-cf9f-4a58-93f4-4b2628daf1ef'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.868159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] The token cache is being invalidate because project 6edb1f2a192247dd891b8e8063a2f8e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:39.868447 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.869064 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:39.870523 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1189/2370] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:39 2022] DELETE /identity/v3/projects/6edb1f2a192247dd891b8e8063a2f8e2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:39.874531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b78fbe-1f88-4b5d-8c2a-35fd760cd4c8 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:39.884093 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b78fbe-1f88-4b5d-8c2a-35fd760cd4c8 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:39.943107 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:39.943816 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:39.944116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:39.944386 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] PATH_INFO: `/v3/users/d97580922e364a8a9ff051c061b2f016` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:39.945547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d97580922e364a8a9ff051c061b2f016)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:39.947506 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:39.999614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd97580922e364a8a9ff051c061b2f016'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.000253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd97580922e364a8a9ff051c061b2f016'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.007075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd97580922e364a8a9ff051c061b2f016'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.015348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd97580922e364a8a9ff051c061b2f016'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.027547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5160447b-95f0-452c-9c4c-31a91606894d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd97580922e364a8a9ff051c061b2f016'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.033583 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1182/2371] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:39 2022] DELETE /identity/v3/users/d97580922e364a8a9ff051c061b2f016 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:40.312521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:40.314092 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:40.327896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e024a-cf9f-4a58-93f4-4b2628daf1ef admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:40.384744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:40.385387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.385664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.385930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] PATH_INFO: `/v3/projects/663c40c676a245ea94b79698fb25fa6e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.387084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=663c40c676a245ea94b79698fb25fa6e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:40.389134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:40.437514 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '663c40c676a245ea94b79698fb25fa6e', 'request_id': 'req-6513dcbc-32fe-42a6-be72-1093af0c8d56'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.443512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '663c40c676a245ea94b79698fb25fa6e', 'request_id': 'req-6513dcbc-32fe-42a6-be72-1093af0c8d56'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.449732 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '663c40c676a245ea94b79698fb25fa6e', 'request_id': 'req-6513dcbc-32fe-42a6-be72-1093af0c8d56'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.464850 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] The token cache is being invalidate because project 663c40c676a245ea94b79698fb25fa6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:40.465119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.465722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6513dcbc-32fe-42a6-be72-1093af0c8d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:40.467264 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1190/2372] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:40 2022] DELETE /identity/v3/projects/663c40c676a245ea94b79698fb25fa6e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:40.472171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a175088-37fd-4995-937a-02c0bba83c5d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.472455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a175088-37fd-4995-937a-02c0bba83c5d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.472727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a175088-37fd-4995-937a-02c0bba83c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.492829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7a175088-37fd-4995-937a-02c0bba83c5d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:40.494035 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7a175088-37fd-4995-937a-02c0bba83c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:40.565088 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1183/2373] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:40.572854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b4be3-afaf-4115-bec8-8dff06cfa84b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.573179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b4be3-afaf-4115-bec8-8dff06cfa84b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.573457 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b4be3-afaf-4115-bec8-8dff06cfa84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.593677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b67b4be3-afaf-4115-bec8-8dff06cfa84b None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:40.594913 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b67b4be3-afaf-4115-bec8-8dff06cfa84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:40.603252 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1191/2374] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:40.607558 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a175088-37fd-4995-937a-02c0bba83c5d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:40.613841 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:40.618146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a175088-37fd-4995-937a-02c0bba83c5d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:40.624424 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e4451-d8f7-4bc4-9e91-4f79b5eebf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:40.625050 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e4451-d8f7-4bc4-9e91-4f79b5eebf9c admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.625322 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e4451-d8f7-4bc4-9e91-4f79b5eebf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.625605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e4451-d8f7-4bc4-9e91-4f79b5eebf9c admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.626328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e4451-d8f7-4bc4-9e91-4f79b5eebf9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:40.626878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e4451-d8f7-4bc4-9e91-4f79b5eebf9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:40.629029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e4451-d8f7-4bc4-9e91-4f79b5eebf9c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:40.640726 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1184/2375] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:40.645065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b4be3-afaf-4115-bec8-8dff06cfa84b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:40.654473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b4be3-afaf-4115-bec8-8dff06cfa84b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:40.660088 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06577624-3a32-489a-8ecf-de253818cfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:40.660699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06577624-3a32-489a-8ecf-de253818cfe4 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.660980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06577624-3a32-489a-8ecf-de253818cfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.661249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06577624-3a32-489a-8ecf-de253818cfe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.661885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06577624-3a32-489a-8ecf-de253818cfe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:40.663849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06577624-3a32-489a-8ecf-de253818cfe4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:40.694978 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1192/2376] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:40 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:40.699183 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e4451-d8f7-4bc4-9e91-4f79b5eebf9c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:40.709641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e4451-d8f7-4bc4-9e91-4f79b5eebf9c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:40.770773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0d048-f6fc-4931-a4d5-8267cbd98fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:40.771394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0d048-f6fc-4931-a4d5-8267cbd98fdf admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.771680 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0d048-f6fc-4931-a4d5-8267cbd98fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.771945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0d048-f6fc-4931-a4d5-8267cbd98fdf admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.772564 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0d048-f6fc-4931-a4d5-8267cbd98fdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:40.774609 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0d048-f6fc-4931-a4d5-8267cbd98fdf admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:40.800891 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1185/2377] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:40 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:40.804974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06577624-3a32-489a-8ecf-de253818cfe4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:40.815705 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06577624-3a32-489a-8ecf-de253818cfe4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:40.820323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b28a-27fd-4eb5-b28d-2b221bb54e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:40.820915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b28a-27fd-4eb5-b28d-2b221bb54e64 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.821194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b28a-27fd-4eb5-b28d-2b221bb54e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.821465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b28a-27fd-4eb5-b28d-2b221bb54e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.822096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39b28a-27fd-4eb5-b28d-2b221bb54e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:40.823709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39b28a-27fd-4eb5-b28d-2b221bb54e64 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:40.833419 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1193/2378] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:40.837157 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0d048-f6fc-4931-a4d5-8267cbd98fdf admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:40.847937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0d048-f6fc-4931-a4d5-8267cbd98fdf admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:40.853561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1636ac-b7be-4e18-999a-ec79b86551c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:40.854152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1636ac-b7be-4e18-999a-ec79b86551c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.854435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1636ac-b7be-4e18-999a-ec79b86551c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.854705 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1636ac-b7be-4e18-999a-ec79b86551c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.855394 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1636ac-b7be-4e18-999a-ec79b86551c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:40.857002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1636ac-b7be-4e18-999a-ec79b86551c6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:40.868034 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1186/2379] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:40.872013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b28a-27fd-4eb5-b28d-2b221bb54e64 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:40.882510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b28a-27fd-4eb5-b28d-2b221bb54e64 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:40.887739 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a25cb7-0cbe-4e59-b996-7630f599b3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:40.888425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a25cb7-0cbe-4e59-b996-7630f599b3e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.888722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a25cb7-0cbe-4e59-b996-7630f599b3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.888982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a25cb7-0cbe-4e59-b996-7630f599b3e0 admin admin] PATH_INFO: `/v3/projects/de71367b6641492aaca226808cf5ab20/users/fd2f260beb48489d99ef0b33100a7bfe/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.903396 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a25cb7-0cbe-4e59-b996-7630f599b3e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de71367b6641492aaca226808cf5ab20, user_id=fd2f260beb48489d99ef0b33100a7bfe, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:40.906036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a25cb7-0cbe-4e59-b996-7630f599b3e0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:40.921051 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1194/2380] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:40 2022] PUT /identity/v3/projects/de71367b6641492aaca226808cf5ab20/users/fd2f260beb48489d99ef0b33100a7bfe/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:40.932610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb2769-db30-4e97-88d6-c40554db1259 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:40.933099 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb2769-db30-4e97-88d6-c40554db1259 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:40.933205 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb2769-db30-4e97-88d6-c40554db1259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:40.961963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-fceb2769-db30-4e97-88d6-c40554db1259 None None] MFA Rules not processed for user `fd2f260beb48489d99ef0b33100a7bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:40.963227 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fceb2769-db30-4e97-88d6-c40554db1259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:41.030104 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1187/2381] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:41.034511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a25cb7-0cbe-4e59-b996-7630f599b3e0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.052698 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:41.099651 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a25cb7-0cbe-4e59-b996-7630f599b3e0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.104683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a68f32-1b4c-41c1-bc89-3f65eb1fdb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.105296 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68f32-1b4c-41c1-bc89-3f65eb1fdb2c admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.105593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68f32-1b4c-41c1-bc89-3f65eb1fdb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.105851 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a68f32-1b4c-41c1-bc89-3f65eb1fdb2c admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.106491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a68f32-1b4c-41c1-bc89-3f65eb1fdb2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.108515 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a68f32-1b4c-41c1-bc89-3f65eb1fdb2c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.141424 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1195/2382] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:41 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:41.145502 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb2769-db30-4e97-88d6-c40554db1259 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.156863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb2769-db30-4e97-88d6-c40554db1259 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.224660 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ece45c-15f6-47ba-a28a-eea4e8f37f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.225255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ece45c-15f6-47ba-a28a-eea4e8f37f50 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.225535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ece45c-15f6-47ba-a28a-eea4e8f37f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.225807 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ece45c-15f6-47ba-a28a-eea4e8f37f50 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.226448 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ece45c-15f6-47ba-a28a-eea4e8f37f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.228533 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ece45c-15f6-47ba-a28a-eea4e8f37f50 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.237143 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:41.256111 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1188/2383] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:41 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:41.257469 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a68f32-1b4c-41c1-bc89-3f65eb1fdb2c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.268053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a68f32-1b4c-41c1-bc89-3f65eb1fdb2c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.272917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac86ed-76d1-45c0-9809-3e99a24ed52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.273661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac86ed-76d1-45c0-9809-3e99a24ed52e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.273919 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac86ed-76d1-45c0-9809-3e99a24ed52e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.274188 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac86ed-76d1-45c0-9809-3e99a24ed52e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.274886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac86ed-76d1-45c0-9809-3e99a24ed52e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.276513 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac86ed-76d1-45c0-9809-3e99a24ed52e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.289922 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1196/2384] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:56:41.293992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ece45c-15f6-47ba-a28a-eea4e8f37f50 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.304965 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ece45c-15f6-47ba-a28a-eea4e8f37f50 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.310053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958ffba-845f-47fb-9402-a9cda8f9f8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.310650 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958ffba-845f-47fb-9402-a9cda8f9f8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.310955 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958ffba-845f-47fb-9402-a9cda8f9f8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.311217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958ffba-845f-47fb-9402-a9cda8f9f8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.311865 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958ffba-845f-47fb-9402-a9cda8f9f8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.313434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958ffba-845f-47fb-9402-a9cda8f9f8e1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.323995 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1189/2385] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:41.330962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac86ed-76d1-45c0-9809-3e99a24ed52e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.341736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac86ed-76d1-45c0-9809-3e99a24ed52e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.347099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae59b2-6d2e-4422-be58-4d4a50bb62e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.347749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae59b2-6d2e-4422-be58-4d4a50bb62e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.348032 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae59b2-6d2e-4422-be58-4d4a50bb62e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.348295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae59b2-6d2e-4422-be58-4d4a50bb62e1 admin admin] PATH_INFO: `/v3/projects/4bdcadf64e734cdb8696d53503f2079d/users/1f01f10d70d349e6a4b17ba78a55e1c8/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.363752 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae59b2-6d2e-4422-be58-4d4a50bb62e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bdcadf64e734cdb8696d53503f2079d, user_id=1f01f10d70d349e6a4b17ba78a55e1c8, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.366690 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae59b2-6d2e-4422-be58-4d4a50bb62e1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.379990 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1197/2386] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:41 2022] PUT /identity/v3/projects/4bdcadf64e734cdb8696d53503f2079d/users/1f01f10d70d349e6a4b17ba78a55e1c8/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:41.390629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b1301-795b-4801-a422-fad353b9aacf None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.391022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b1301-795b-4801-a422-fad353b9aacf None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.391326 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b1301-795b-4801-a422-fad353b9aacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.422510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a79b1301-795b-4801-a422-fad353b9aacf None None] MFA Rules not processed for user `1f01f10d70d349e6a4b17ba78a55e1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:41.424299 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a79b1301-795b-4801-a422-fad353b9aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:41.500430 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1190/2387] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:41.507182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae59b2-6d2e-4422-be58-4d4a50bb62e1 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.579496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae59b2-6d2e-4422-be58-4d4a50bb62e1 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.584983 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8539bc-f8f4-4b8a-8304-5f63999fe93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.585594 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8539bc-f8f4-4b8a-8304-5f63999fe93b admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.585869 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8539bc-f8f4-4b8a-8304-5f63999fe93b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.586140 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8539bc-f8f4-4b8a-8304-5f63999fe93b admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.586810 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8539bc-f8f4-4b8a-8304-5f63999fe93b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.588824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8539bc-f8f4-4b8a-8304-5f63999fe93b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.620557 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1198/2388] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:41 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:41.621715 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b1301-795b-4801-a422-fad353b9aacf None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.632460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b1301-795b-4801-a422-fad353b9aacf None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.718603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce986b-ca9c-459e-ac21-d819e0ecedd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.719560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce986b-ca9c-459e-ac21-d819e0ecedd5 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.720036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce986b-ca9c-459e-ac21-d819e0ecedd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.720525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce986b-ca9c-459e-ac21-d819e0ecedd5 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.721728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce986b-ca9c-459e-ac21-d819e0ecedd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.725034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce986b-ca9c-459e-ac21-d819e0ecedd5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.749328 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1191/2389] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:41 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:41.753858 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8539bc-f8f4-4b8a-8304-5f63999fe93b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.764818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8539bc-f8f4-4b8a-8304-5f63999fe93b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.770114 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94d1a-7eaa-4311-9053-b0095e257842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.771082 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94d1a-7eaa-4311-9053-b0095e257842 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.771462 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94d1a-7eaa-4311-9053-b0095e257842 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.771738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94d1a-7eaa-4311-9053-b0095e257842 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.772388 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94d1a-7eaa-4311-9053-b0095e257842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.774974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94d1a-7eaa-4311-9053-b0095e257842 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.791726 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1199/2390] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:41.797780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce986b-ca9c-459e-ac21-d819e0ecedd5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.815002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce986b-ca9c-459e-ac21-d819e0ecedd5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.824631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fd2ea-fc31-442c-b054-a0db1537da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.825626 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fd2ea-fc31-442c-b054-a0db1537da0b admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.825913 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fd2ea-fc31-442c-b054-a0db1537da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.826186 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fd2ea-fc31-442c-b054-a0db1537da0b admin admin] PATH_INFO: `/v3/projects/3cdd06ee91e14f4b84e259c8c4a7409c/users/198de602ecd8462e91f8392ffa3224b4/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.844136 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fd2ea-fc31-442c-b054-a0db1537da0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cdd06ee91e14f4b84e259c8c4a7409c, user_id=198de602ecd8462e91f8392ffa3224b4, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.846819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fd2ea-fc31-442c-b054-a0db1537da0b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.868099 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1192/2391] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:41 2022] PUT /identity/v3/projects/3cdd06ee91e14f4b84e259c8c4a7409c/users/198de602ecd8462e91f8392ffa3224b4/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:41.873742 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94d1a-7eaa-4311-9053-b0095e257842 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.884889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94d1a-7eaa-4311-9053-b0095e257842 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.924774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:41.944804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038765a6-eb1e-4d55-ab15-ab65d43fba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.945412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038765a6-eb1e-4d55-ab15-ab65d43fba5f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.945799 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038765a6-eb1e-4d55-ab15-ab65d43fba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.946071 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038765a6-eb1e-4d55-ab15-ab65d43fba5f admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.946849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038765a6-eb1e-4d55-ab15-ab65d43fba5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.948653 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038765a6-eb1e-4d55-ab15-ab65d43fba5f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:41.959138 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1200/2392] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:41.964316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fd2ea-fc31-442c-b054-a0db1537da0b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:41.975656 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fd2ea-fc31-442c-b054-a0db1537da0b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:41.981966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55237b5b-a106-473f-840f-a6ad77554591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:41.982619 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55237b5b-a106-473f-840f-a6ad77554591 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:41.982931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55237b5b-a106-473f-840f-a6ad77554591 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:41.983207 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55237b5b-a106-473f-840f-a6ad77554591 admin admin] PATH_INFO: `/v3/projects/3cdd06ee91e14f4b84e259c8c4a7409c/users/198de602ecd8462e91f8392ffa3224b4/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:41.985519 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55237b5b-a106-473f-840f-a6ad77554591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cdd06ee91e14f4b84e259c8c4a7409c, user_id=198de602ecd8462e91f8392ffa3224b4, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:41.988163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55237b5b-a106-473f-840f-a6ad77554591 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:42.000224 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1193/2393] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:41 2022] PUT /identity/v3/projects/3cdd06ee91e14f4b84e259c8c4a7409c/users/198de602ecd8462e91f8392ffa3224b4/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:42.010685 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cad9c-db96-4bbb-b6d3-ca8a8ede59e1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:42.011023 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cad9c-db96-4bbb-b6d3-ca8a8ede59e1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:42.011298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cad9c-db96-4bbb-b6d3-ca8a8ede59e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:42.040294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8e5cad9c-db96-4bbb-b6d3-ca8a8ede59e1 None None] MFA Rules not processed for user `198de602ecd8462e91f8392ffa3224b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:42.041511 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8e5cad9c-db96-4bbb-b6d3-ca8a8ede59e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:42.121926 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1201/2394] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:42.134371 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55237b5b-a106-473f-840f-a6ad77554591 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:42.136485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-55237b5b-a106-473f-840f-a6ad77554591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:42.161047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:42.201214 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55237b5b-a106-473f-840f-a6ad77554591 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:42.205216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50234071-1900-4be4-87c3-ee966b0fe330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:42.205815 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50234071-1900-4be4-87c3-ee966b0fe330 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:42.206113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50234071-1900-4be4-87c3-ee966b0fe330 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:42.206351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50234071-1900-4be4-87c3-ee966b0fe330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:42.217611 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50234071-1900-4be4-87c3-ee966b0fe330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:42.219079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50234071-1900-4be4-87c3-ee966b0fe330 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:42.228795 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1194/2395] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:42.272461 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cad9c-db96-4bbb-b6d3-ca8a8ede59e1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:42.274160 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8e5cad9c-db96-4bbb-b6d3-ca8a8ede59e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:42.289512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cad9c-db96-4bbb-b6d3-ca8a8ede59e1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:42.294092 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a166b-65f9-4d0d-b511-378a421c18f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:42.294735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a166b-65f9-4d0d-b511-378a421c18f8 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:42.295034 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a166b-65f9-4d0d-b511-378a421c18f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:42.295359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a166b-65f9-4d0d-b511-378a421c18f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:42.357936 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a166b-65f9-4d0d-b511-378a421c18f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:42.359849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a166b-65f9-4d0d-b511-378a421c18f8 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:42.373407 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1202/2396] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:45.433605 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50234071-1900-4be4-87c3-ee966b0fe330 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:45.520971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50234071-1900-4be4-87c3-ee966b0fe330 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:45.526530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:45.527287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:45.527631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:45.528083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] PATH_INFO: `/v3/users/fd2f260beb48489d99ef0b33100a7bfe` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:45.529429 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd2f260beb48489d99ef0b33100a7bfe)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:45.531435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:45.588623 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd2f260beb48489d99ef0b33100a7bfe'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:45.589372 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd2f260beb48489d99ef0b33100a7bfe'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:45.598149 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd2f260beb48489d99ef0b33100a7bfe'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:45.610115 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd2f260beb48489d99ef0b33100a7bfe'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:45.631997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd2f260beb48489d99ef0b33100a7bfe'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:45.644295 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1195/2397] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:45 2022] DELETE /identity/v3/users/fd2f260beb48489d99ef0b33100a7bfe => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:45.649730 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c69b4c-a92e-4706-a47a-c82b59648495 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:45.650831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c69b4c-a92e-4706-a47a-c82b59648495 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:45.651289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c69b4c-a92e-4706-a47a-c82b59648495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:45.687222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-56c69b4c-a92e-4706-a47a-c82b59648495 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:45.688825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-56c69b4c-a92e-4706-a47a-c82b59648495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:45.761908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:45.772340 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1203/2398] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:56:45.782362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:45.783932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:45.847585 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:45.861513 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410098f6-d584-442e-8fd2-f5e32041369f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:45.866644 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd988042-d5ac-430a-b165-00e91f9bf616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:45.867272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd988042-d5ac-430a-b165-00e91f9bf616 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:45.867903 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd988042-d5ac-430a-b165-00e91f9bf616 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:45.868293 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd988042-d5ac-430a-b165-00e91f9bf616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:45.877912 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd988042-d5ac-430a-b165-00e91f9bf616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:45.879384 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd988042-d5ac-430a-b165-00e91f9bf616 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:45.889603 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1196/2399] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:46.194743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c69b4c-a92e-4706-a47a-c82b59648495 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:46.196238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-56c69b4c-a92e-4706-a47a-c82b59648495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:46.211392 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c69b4c-a92e-4706-a47a-c82b59648495 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:46.216153 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:46.216805 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:46.217078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:46.217347 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] PATH_INFO: `/v3/projects/de71367b6641492aaca226808cf5ab20` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:46.218664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=de71367b6641492aaca226808cf5ab20)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:46.220599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:46.296211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de71367b6641492aaca226808cf5ab20', 'request_id': 'req-ec417ad8-219c-4622-9b27-8f7b374339cf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.302681 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de71367b6641492aaca226808cf5ab20', 'request_id': 'req-ec417ad8-219c-4622-9b27-8f7b374339cf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.310346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de71367b6641492aaca226808cf5ab20', 'request_id': 'req-ec417ad8-219c-4622-9b27-8f7b374339cf'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.328359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] The token cache is being invalidate because project de71367b6641492aaca226808cf5ab20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:46.328650 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.329266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.330959 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1204/2400] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:46 2022] DELETE /identity/v3/projects/de71367b6641492aaca226808cf5ab20 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:46.334676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd988042-d5ac-430a-b165-00e91f9bf616 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:46.345800 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd988042-d5ac-430a-b165-00e91f9bf616 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:46.412621 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:46.413314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:46.413657 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:46.413997 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] PATH_INFO: `/v3/users/1f01f10d70d349e6a4b17ba78a55e1c8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:46.415257 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f01f10d70d349e6a4b17ba78a55e1c8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:46.417449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:46.477469 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f01f10d70d349e6a4b17ba78a55e1c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.478191 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f01f10d70d349e6a4b17ba78a55e1c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.486321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f01f10d70d349e6a4b17ba78a55e1c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.499232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f01f10d70d349e6a4b17ba78a55e1c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.529281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f01f10d70d349e6a4b17ba78a55e1c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:46.538032 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1197/2401] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:46 2022] DELETE /identity/v3/users/1f01f10d70d349e6a4b17ba78a55e1c8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:47.185314 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:47.187775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:47.204839 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec417ad8-219c-4622-9b27-8f7b374339cf admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:47.263422 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:47.264060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:47.264338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:47.264604 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] PATH_INFO: `/v3/projects/4bdcadf64e734cdb8696d53503f2079d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:47.265738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bdcadf64e734cdb8696d53503f2079d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:47.267673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:47.314458 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bdcadf64e734cdb8696d53503f2079d', 'request_id': 'req-fc57dfb8-8127-4f1a-85ba-05543d04e349'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.320882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bdcadf64e734cdb8696d53503f2079d', 'request_id': 'req-fc57dfb8-8127-4f1a-85ba-05543d04e349'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.327381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bdcadf64e734cdb8696d53503f2079d', 'request_id': 'req-fc57dfb8-8127-4f1a-85ba-05543d04e349'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.342068 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] The token cache is being invalidate because project 4bdcadf64e734cdb8696d53503f2079d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:47.342359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.343016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.344530 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1205/2402] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:47 2022] DELETE /identity/v3/projects/4bdcadf64e734cdb8696d53503f2079d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:56:47.348740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:47.350177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:47.363754 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001c506-89ca-44a5-9ab7-b259592ceab9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:47.423316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:47.424254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:47.424527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:47.424792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] PATH_INFO: `/v3/users/198de602ecd8462e91f8392ffa3224b4` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:47.426007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=198de602ecd8462e91f8392ffa3224b4)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:47.427914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:47.432565 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:47.475527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198de602ecd8462e91f8392ffa3224b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.476405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198de602ecd8462e91f8392ffa3224b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.487353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198de602ecd8462e91f8392ffa3224b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.498687 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198de602ecd8462e91f8392ffa3224b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.511078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d1d3223a-4785-42ad-bf6d-efa10818ccf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198de602ecd8462e91f8392ffa3224b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.517142 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1198/2403] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:47 2022] DELETE /identity/v3/users/198de602ecd8462e91f8392ffa3224b4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:47.800971 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:47.802391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:47.817806 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57dfb8-8127-4f1a-85ba-05543d04e349 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:47.828656 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:47.876815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:47.877438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:47.877715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:47.877984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] PATH_INFO: `/v3/projects/3cdd06ee91e14f4b84e259c8c4a7409c` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:47.879141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cdd06ee91e14f4b84e259c8c4a7409c)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:47.881061 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:47.928865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cdd06ee91e14f4b84e259c8c4a7409c', 'request_id': 'req-69b7ba90-8760-437e-9070-1212615fda26'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.934168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cdd06ee91e14f4b84e259c8c4a7409c', 'request_id': 'req-69b7ba90-8760-437e-9070-1212615fda26'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.940321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cdd06ee91e14f4b84e259c8c4a7409c', 'request_id': 'req-69b7ba90-8760-437e-9070-1212615fda26'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.953299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] The token cache is being invalidate because project 3cdd06ee91e14f4b84e259c8c4a7409c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:47.953586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.954269 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-69b7ba90-8760-437e-9070-1212615fda26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:47.955818 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1206/2404] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:47 2022] DELETE /identity/v3/projects/3cdd06ee91e14f4b84e259c8c4a7409c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:56:47.962152 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e6dea-c9b7-46f1-a230-3dc803c29f34 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:47.962439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e6dea-c9b7-46f1-a230-3dc803c29f34 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:47.962735 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e6dea-c9b7-46f1-a230-3dc803c29f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:47.983796 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7c5e6dea-c9b7-46f1-a230-3dc803c29f34 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:47.985008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7c5e6dea-c9b7-46f1-a230-3dc803c29f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:48.059194 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1199/2405] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:48.072381 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf19f4-bec0-44a5-b763-2bc7edad7a6c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.072798 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf19f4-bec0-44a5-b763-2bc7edad7a6c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.073189 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf19f4-bec0-44a5-b763-2bc7edad7a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.098470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-eacf19f4-bec0-44a5-b763-2bc7edad7a6c None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:48.099710 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-eacf19f4-bec0-44a5-b763-2bc7edad7a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:48.108597 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1207/2406] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:48.113188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e6dea-c9b7-46f1-a230-3dc803c29f34 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:48.123360 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e6dea-c9b7-46f1-a230-3dc803c29f34 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:48.128291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735050b3-8541-4bc5-89ca-5c85c8ecb3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:48.128980 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735050b3-8541-4bc5-89ca-5c85c8ecb3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.129258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735050b3-8541-4bc5-89ca-5c85c8ecb3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.129531 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735050b3-8541-4bc5-89ca-5c85c8ecb3d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.130171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735050b3-8541-4bc5-89ca-5c85c8ecb3d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:48.130478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735050b3-8541-4bc5-89ca-5c85c8ecb3d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:48.132164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735050b3-8541-4bc5-89ca-5c85c8ecb3d8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:48.144293 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1200/2407] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:48.148710 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf19f4-bec0-44a5-b763-2bc7edad7a6c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:48.159438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf19f4-bec0-44a5-b763-2bc7edad7a6c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:48.164708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd755a1-b045-46b9-8f34-622041cea01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:48.165308 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd755a1-b045-46b9-8f34-622041cea01e admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.165593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd755a1-b045-46b9-8f34-622041cea01e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.165872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd755a1-b045-46b9-8f34-622041cea01e admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.166506 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd755a1-b045-46b9-8f34-622041cea01e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:48.168462 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd755a1-b045-46b9-8f34-622041cea01e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:48.202426 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1208/2408] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:48 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:48.203614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735050b3-8541-4bc5-89ca-5c85c8ecb3d8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:48.213884 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735050b3-8541-4bc5-89ca-5c85c8ecb3d8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:48.296772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5dbea-0274-43b0-8909-3b9b60bdef26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:48.297466 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5dbea-0274-43b0-8909-3b9b60bdef26 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.297842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5dbea-0274-43b0-8909-3b9b60bdef26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.298133 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5dbea-0274-43b0-8909-3b9b60bdef26 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.299146 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5dbea-0274-43b0-8909-3b9b60bdef26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:48.301143 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5dbea-0274-43b0-8909-3b9b60bdef26 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:48.332665 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:48.341573 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1201/2409] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:48 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:48.342818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd755a1-b045-46b9-8f34-622041cea01e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:48.354378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd755a1-b045-46b9-8f34-622041cea01e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:48.360640 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43d45-4290-4577-9137-44887a7176d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:48.361273 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43d45-4290-4577-9137-44887a7176d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.361600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43d45-4290-4577-9137-44887a7176d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.361802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43d45-4290-4577-9137-44887a7176d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.362444 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d43d45-4290-4577-9137-44887a7176d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:48.364066 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d43d45-4290-4577-9137-44887a7176d2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:48.377572 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1209/2410] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:56:48.378998 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5dbea-0274-43b0-8909-3b9b60bdef26 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:48.389610 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5dbea-0274-43b0-8909-3b9b60bdef26 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:48.394678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bbaa4-3c6f-4976-a474-1da91d90e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:48.395368 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bbaa4-3c6f-4976-a474-1da91d90e4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.395762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bbaa4-3c6f-4976-a474-1da91d90e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.396219 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bbaa4-3c6f-4976-a474-1da91d90e4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.397043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bbaa4-3c6f-4976-a474-1da91d90e4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:48.398819 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bbaa4-3c6f-4976-a474-1da91d90e4b7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:48.409994 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1202/2411] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:48.414412 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43d45-4290-4577-9137-44887a7176d2 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:48.424824 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43d45-4290-4577-9137-44887a7176d2 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:48.429791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93376e-28c4-410c-becb-f788ad9b4100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:48.430473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93376e-28c4-410c-becb-f788ad9b4100 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.430772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93376e-28c4-410c-becb-f788ad9b4100 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.431092 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93376e-28c4-410c-becb-f788ad9b4100 admin admin] PATH_INFO: `/v3/projects/7c3e75da17c440bb968ea00335dd46c1/users/685e99b16601418cb54ff66d468e8e9f/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.446191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93376e-28c4-410c-becb-f788ad9b4100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c3e75da17c440bb968ea00335dd46c1, user_id=685e99b16601418cb54ff66d468e8e9f, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:48.448957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93376e-28c4-410c-becb-f788ad9b4100 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:48.463627 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1210/2412] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:48 2022] PUT /identity/v3/projects/7c3e75da17c440bb968ea00335dd46c1/users/685e99b16601418cb54ff66d468e8e9f/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:48.475011 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfe02-6142-419f-b702-9dfe3522a00b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.475371 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfe02-6142-419f-b702-9dfe3522a00b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.475652 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfe02-6142-419f-b702-9dfe3522a00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.503932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-df7bfe02-6142-419f-b702-9dfe3522a00b None None] MFA Rules not processed for user `685e99b16601418cb54ff66d468e8e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:48.505160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-df7bfe02-6142-419f-b702-9dfe3522a00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:48.572045 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1203/2413] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:48.584033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93376e-28c4-410c-becb-f788ad9b4100 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:48.585923 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2a93376e-28c4-410c-becb-f788ad9b4100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:48.630604 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:48.647525 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93376e-28c4-410c-becb-f788ad9b4100 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:48.653992 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabeb6a-161b-40b8-b292-deec8890aaf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:48.654770 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabeb6a-161b-40b8-b292-deec8890aaf2 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.654955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabeb6a-161b-40b8-b292-deec8890aaf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.655221 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabeb6a-161b-40b8-b292-deec8890aaf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.666108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabeb6a-161b-40b8-b292-deec8890aaf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:48.667725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabeb6a-161b-40b8-b292-deec8890aaf2 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:48.677779 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1211/2414] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:48.697549 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfe02-6142-419f-b702-9dfe3522a00b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:48.762854 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfe02-6142-419f-b702-9dfe3522a00b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:48.767871 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:48.768498 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.768777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.769048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] PATH_INFO: `/v3/users/685e99b16601418cb54ff66d468e8e9f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.770174 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=685e99b16601418cb54ff66d468e8e9f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:48.772129 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:48.827395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685e99b16601418cb54ff66d468e8e9f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:48.828051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685e99b16601418cb54ff66d468e8e9f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:48.835524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685e99b16601418cb54ff66d468e8e9f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:48.844462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685e99b16601418cb54ff66d468e8e9f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:48.856603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685e99b16601418cb54ff66d468e8e9f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:48.858082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:48.864062 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1204/2415] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:48 2022] DELETE /identity/v3/users/685e99b16601418cb54ff66d468e8e9f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:48.868825 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116a8a-ea5b-4729-8928-d9ba768c21a2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:48.869158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116a8a-ea5b-4729-8928-d9ba768c21a2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:48.869434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116a8a-ea5b-4729-8928-d9ba768c21a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:48.889224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d1116a8a-ea5b-4729-8928-d9ba768c21a2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:48.890463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d1116a8a-ea5b-4729-8928-d9ba768c21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:48.953061 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1212/2416] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:48.963510 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:48.965124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:49.024155 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484506d0-f590-4bf1-95b2-a1866aacab84 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:49.028239 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b46164-022f-44da-85a2-542105cc3d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:49.028870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b46164-022f-44da-85a2-542105cc3d4b service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.029150 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b46164-022f-44da-85a2-542105cc3d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.029407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b46164-022f-44da-85a2-542105cc3d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.038620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b46164-022f-44da-85a2-542105cc3d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:49.040076 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b46164-022f-44da-85a2-542105cc3d4b service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:49.049670 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1205/2417] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:49.336019 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1116a8a-ea5b-4729-8928-d9ba768c21a2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:49.337673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d1116a8a-ea5b-4729-8928-d9ba768c21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:49.351384 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1116a8a-ea5b-4729-8928-d9ba768c21a2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:49.356822 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:49.357809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.358246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.358657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] PATH_INFO: `/v3/projects/7c3e75da17c440bb968ea00335dd46c1` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.361362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c3e75da17c440bb968ea00335dd46c1)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:49.363697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:49.419715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c3e75da17c440bb968ea00335dd46c1', 'request_id': 'req-751e0b7a-bb3f-4615-be36-49bcbd682960'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:49.428633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c3e75da17c440bb968ea00335dd46c1', 'request_id': 'req-751e0b7a-bb3f-4615-be36-49bcbd682960'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:49.437594 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c3e75da17c440bb968ea00335dd46c1', 'request_id': 'req-751e0b7a-bb3f-4615-be36-49bcbd682960'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:49.456394 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] The token cache is being invalidate because project 7c3e75da17c440bb968ea00335dd46c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:49.456699 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:49.457400 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:49.459011 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1213/2418] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:49 2022] DELETE /identity/v3/projects/7c3e75da17c440bb968ea00335dd46c1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:49.464286 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332d155-74c2-4e5a-bbd7-ac604014ae0d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.465462 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332d155-74c2-4e5a-bbd7-ac604014ae0d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.465927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332d155-74c2-4e5a-bbd7-ac604014ae0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.492196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3332d155-74c2-4e5a-bbd7-ac604014ae0d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:49.493676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3332d155-74c2-4e5a-bbd7-ac604014ae0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:49.550628 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:49.552177 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:49.578760 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1206/2419] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:49.585247 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca31f0-1dd8-473b-8a65-41022dadd7cc None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.585771 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca31f0-1dd8-473b-8a65-41022dadd7cc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.586050 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca31f0-1dd8-473b-8a65-41022dadd7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.608562 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-74ca31f0-1dd8-473b-8a65-41022dadd7cc None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:49.609782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-74ca31f0-1dd8-473b-8a65-41022dadd7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:49.621927 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1207/2420] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:49.624792 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e0b7a-bb3f-4615-be36-49bcbd682960 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:49.626071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca31f0-1dd8-473b-8a65-41022dadd7cc None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:49.630980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5d92d-8ee6-4568-b92f-16fa87bc2ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:49.632046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5d92d-8ee6-4568-b92f-16fa87bc2ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.632181 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5d92d-8ee6-4568-b92f-16fa87bc2ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.632420 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5d92d-8ee6-4568-b92f-16fa87bc2ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.634548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-28b5d92d-8ee6-4568-b92f-16fa87bc2ee6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:49.635934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca31f0-1dd8-473b-8a65-41022dadd7cc None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:49.647472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050ae45-0cd2-4686-b744-9b3b8bef5716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:49.647976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050ae45-0cd2-4686-b744-9b3b8bef5716 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.648252 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050ae45-0cd2-4686-b744-9b3b8bef5716 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.648516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050ae45-0cd2-4686-b744-9b3b8bef5716 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.650794 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050ae45-0cd2-4686-b744-9b3b8bef5716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:49.651365 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050ae45-0cd2-4686-b744-9b3b8bef5716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:49.651937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5d92d-8ee6-4568-b92f-16fa87bc2ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:49.652981 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050ae45-0cd2-4686-b744-9b3b8bef5716 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:49.653735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5d92d-8ee6-4568-b92f-16fa87bc2ee6 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:49.670019 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1208/2421] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:49.670745 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1214/2422] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:56:49.672485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050ae45-0cd2-4686-b744-9b3b8bef5716 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:49.684077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050ae45-0cd2-4686-b744-9b3b8bef5716 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:49.689047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fd396-ec04-41dd-99a4-c6c55228670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:49.689662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fd396-ec04-41dd-99a4-c6c55228670d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.689943 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fd396-ec04-41dd-99a4-c6c55228670d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.690218 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fd396-ec04-41dd-99a4-c6c55228670d admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.690937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fd396-ec04-41dd-99a4-c6c55228670d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:49.692986 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fd396-ec04-41dd-99a4-c6c55228670d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:49.725375 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1209/2423] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:49 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:56:49.729739 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5d92d-8ee6-4568-b92f-16fa87bc2ee6 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:49.740303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5d92d-8ee6-4568-b92f-16fa87bc2ee6 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:49.745238 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:49.810518 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31c90f-43e8-4d4b-8949-5a0c88d29e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:49.811277 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31c90f-43e8-4d4b-8949-5a0c88d29e36 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.811559 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31c90f-43e8-4d4b-8949-5a0c88d29e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.811860 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31c90f-43e8-4d4b-8949-5a0c88d29e36 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.812517 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31c90f-43e8-4d4b-8949-5a0c88d29e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:49.814578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31c90f-43e8-4d4b-8949-5a0c88d29e36 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:49.836868 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1215/2424] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:49 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:49.841123 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fd396-ec04-41dd-99a4-c6c55228670d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:49.852401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fd396-ec04-41dd-99a4-c6c55228670d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:49.857996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4be42-1b3e-4f75-b103-642ea9b04dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:49.858704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4be42-1b3e-4f75-b103-642ea9b04dde admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.859082 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4be42-1b3e-4f75-b103-642ea9b04dde admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.859417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4be42-1b3e-4f75-b103-642ea9b04dde admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.860131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4be42-1b3e-4f75-b103-642ea9b04dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:49.861789 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4be42-1b3e-4f75-b103-642ea9b04dde admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:49.872641 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1210/2425] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:56:49.876533 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31c90f-43e8-4d4b-8949-5a0c88d29e36 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:49.887368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31c90f-43e8-4d4b-8949-5a0c88d29e36 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:49.893293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa379f0f-bb57-4d41-93a6-d2a3f91783aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:49.893920 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa379f0f-bb57-4d41-93a6-d2a3f91783aa admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.894185 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa379f0f-bb57-4d41-93a6-d2a3f91783aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.894463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa379f0f-bb57-4d41-93a6-d2a3f91783aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.895200 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa379f0f-bb57-4d41-93a6-d2a3f91783aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:49.896753 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa379f0f-bb57-4d41-93a6-d2a3f91783aa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:49.907611 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1216/2426] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:49.911976 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4be42-1b3e-4f75-b103-642ea9b04dde admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:49.922755 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4be42-1b3e-4f75-b103-642ea9b04dde admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:49.928260 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb234bf-52a1-4bda-909d-0798b76dc1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:49.928916 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb234bf-52a1-4bda-909d-0798b76dc1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:49.929198 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb234bf-52a1-4bda-909d-0798b76dc1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:49.929468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb234bf-52a1-4bda-909d-0798b76dc1a9 admin admin] PATH_INFO: `/v3/projects/200fe485e7ce478cbd03c3fb62b24d6f/users/d2478b9d83f440bf95cb104f0417f7a7/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:49.945216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb234bf-52a1-4bda-909d-0798b76dc1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200fe485e7ce478cbd03c3fb62b24d6f, user_id=d2478b9d83f440bf95cb104f0417f7a7, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:49.947904 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb234bf-52a1-4bda-909d-0798b76dc1a9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:49.963782 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1211/2427] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:49 2022] PUT /identity/v3/projects/200fe485e7ce478cbd03c3fb62b24d6f/users/d2478b9d83f440bf95cb104f0417f7a7/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:50.033212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435750d-8b5b-4efa-be65-25d07b05b698 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:50.033500 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435750d-8b5b-4efa-be65-25d07b05b698 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:50.033774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435750d-8b5b-4efa-be65-25d07b05b698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:50.064932 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4435750d-8b5b-4efa-be65-25d07b05b698 None None] MFA Rules not processed for user `d2478b9d83f440bf95cb104f0417f7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:50.066157 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4435750d-8b5b-4efa-be65-25d07b05b698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:50.132218 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1217/2428] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:56:50.143445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb234bf-52a1-4bda-909d-0798b76dc1a9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:50.145460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2fb234bf-52a1-4bda-909d-0798b76dc1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:50.175168 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:50.205836 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb234bf-52a1-4bda-909d-0798b76dc1a9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:50.210655 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c22eed-83cd-45ac-9c4a-6e4819a9b3e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:50.211384 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c22eed-83cd-45ac-9c4a-6e4819a9b3e5 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:50.211730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c22eed-83cd-45ac-9c4a-6e4819a9b3e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:50.212349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c22eed-83cd-45ac-9c4a-6e4819a9b3e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:50.223488 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c22eed-83cd-45ac-9c4a-6e4819a9b3e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:50.224949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c22eed-83cd-45ac-9c4a-6e4819a9b3e5 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:50.235685 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1212/2429] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:52.900053 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435750d-8b5b-4efa-be65-25d07b05b698 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:52.901638 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4435750d-8b5b-4efa-be65-25d07b05b698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:52.978184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435750d-8b5b-4efa-be65-25d07b05b698 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:52.982830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:52.987642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:52.987642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:52.987642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] PATH_INFO: `/v3/users/43af1b9b66804624aff6eef2f435d581` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:52.987642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43af1b9b66804624aff6eef2f435d581)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:52.987642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:53.017028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:53.103161 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43af1b9b66804624aff6eef2f435d581'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.111657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43af1b9b66804624aff6eef2f435d581'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.112632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43af1b9b66804624aff6eef2f435d581'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.125133 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43af1b9b66804624aff6eef2f435d581'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.166391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b0852fd3-b59d-47eb-91e6-506d95d52612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43af1b9b66804624aff6eef2f435d581'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.182567 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1218/2430] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:52 2022] DELETE /identity/v3/users/43af1b9b66804624aff6eef2f435d581 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:53.458781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c22eed-83cd-45ac-9c4a-6e4819a9b3e5 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:53.475767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c22eed-83cd-45ac-9c4a-6e4819a9b3e5 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:53.581221 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:53.581894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:53.582134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:53.582412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] PATH_INFO: `/v3/projects/5bca750f880a44988f7324b2f8da3e7b` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:53.583782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bca750f880a44988f7324b2f8da3e7b)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:53.586318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:53.654817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bca750f880a44988f7324b2f8da3e7b', 'request_id': 'req-93eacfb4-4cea-4ac2-aad6-70db8897f95d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.660762 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bca750f880a44988f7324b2f8da3e7b', 'request_id': 'req-93eacfb4-4cea-4ac2-aad6-70db8897f95d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.668599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bca750f880a44988f7324b2f8da3e7b', 'request_id': 'req-93eacfb4-4cea-4ac2-aad6-70db8897f95d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.683889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] The token cache is being invalidate because project 5bca750f880a44988f7324b2f8da3e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:53.684217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.684820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-93eacfb4-4cea-4ac2-aad6-70db8897f95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:53.686699 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1213/2431] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:53 2022] DELETE /identity/v3/projects/5bca750f880a44988f7324b2f8da3e7b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:53.691731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9aa94-a6fa-42cb-81a5-6a31329d5472 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:53.692015 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9aa94-a6fa-42cb-81a5-6a31329d5472 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:53.692289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9aa94-a6fa-42cb-81a5-6a31329d5472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:53.713395 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-03e9aa94-a6fa-42cb-81a5-6a31329d5472 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:53.714985 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-03e9aa94-a6fa-42cb-81a5-6a31329d5472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:53.775228 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1219/2432] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:56:53.784016 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387a420-711c-42c6-be2a-3af02257b330 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:53.784297 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387a420-711c-42c6-be2a-3af02257b330 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:53.784606 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387a420-711c-42c6-be2a-3af02257b330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:53.808133 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8387a420-711c-42c6-be2a-3af02257b330 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:53.810056 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8387a420-711c-42c6-be2a-3af02257b330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:53.818573 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1214/2433] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:53.822899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9aa94-a6fa-42cb-81a5-6a31329d5472 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:53.834310 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9aa94-a6fa-42cb-81a5-6a31329d5472 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:53.840062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2746d-d889-408c-b110-62af2780e218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:53.840224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2746d-d889-408c-b110-62af2780e218 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:53.840482 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2746d-d889-408c-b110-62af2780e218 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:53.840818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2746d-d889-408c-b110-62af2780e218 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:53.841544 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2746d-d889-408c-b110-62af2780e218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:53.841914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2746d-d889-408c-b110-62af2780e218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:53.843626 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2746d-d889-408c-b110-62af2780e218 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:53.855849 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1220/2434] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:53.864287 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387a420-711c-42c6-be2a-3af02257b330 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:53.873599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387a420-711c-42c6-be2a-3af02257b330 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:53.878161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65cdd-5450-42ec-8d5e-c35664f56de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:53.878786 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65cdd-5450-42ec-8d5e-c35664f56de4 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:53.879064 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65cdd-5450-42ec-8d5e-c35664f56de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:53.879333 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65cdd-5450-42ec-8d5e-c35664f56de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:53.879966 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb65cdd-5450-42ec-8d5e-c35664f56de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:53.881954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb65cdd-5450-42ec-8d5e-c35664f56de4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:53.910626 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1215/2435] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:53 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:53.914900 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2746d-d889-408c-b110-62af2780e218 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:53.926740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2746d-d889-408c-b110-62af2780e218 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:53.982847 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11d59f-2ea4-4d9f-8e16-01a1776bf627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:53.983472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11d59f-2ea4-4d9f-8e16-01a1776bf627 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:53.983760 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11d59f-2ea4-4d9f-8e16-01a1776bf627 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:53.984036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11d59f-2ea4-4d9f-8e16-01a1776bf627 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:53.984667 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11d59f-2ea4-4d9f-8e16-01a1776bf627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:53.986586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11d59f-2ea4-4d9f-8e16-01a1776bf627 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.013756 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1221/2436] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:53 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:54.020397 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65cdd-5450-42ec-8d5e-c35664f56de4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.034141 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65cdd-5450-42ec-8d5e-c35664f56de4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.039162 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af5313-26f2-4df4-ab6d-b77b0335b9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.039751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af5313-26f2-4df4-ab6d-b77b0335b9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.040031 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af5313-26f2-4df4-ab6d-b77b0335b9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.040279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af5313-26f2-4df4-ab6d-b77b0335b9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.040933 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af5313-26f2-4df4-ab6d-b77b0335b9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.042538 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af5313-26f2-4df4-ab6d-b77b0335b9c5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.052968 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1216/2437] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:56:54.057021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11d59f-2ea4-4d9f-8e16-01a1776bf627 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.067137 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11d59f-2ea4-4d9f-8e16-01a1776bf627 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.072021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2a91d-b571-4f3c-b0bb-c2ee7bc0b192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.072611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2a91d-b571-4f3c-b0bb-c2ee7bc0b192 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.072888 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2a91d-b571-4f3c-b0bb-c2ee7bc0b192 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.073155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2a91d-b571-4f3c-b0bb-c2ee7bc0b192 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.073801 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2a91d-b571-4f3c-b0bb-c2ee7bc0b192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.075460 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2a91d-b571-4f3c-b0bb-c2ee7bc0b192 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.088013 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1222/2438] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:54.092951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af5313-26f2-4df4-ab6d-b77b0335b9c5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.103926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af5313-26f2-4df4-ab6d-b77b0335b9c5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.109520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902385f7-1a35-46e3-96ef-a1c20bb55f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.110158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902385f7-1a35-46e3-96ef-a1c20bb55f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.110441 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902385f7-1a35-46e3-96ef-a1c20bb55f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.110736 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902385f7-1a35-46e3-96ef-a1c20bb55f96 admin admin] PATH_INFO: `/v3/projects/3fa0885d35c84ca3ae116dcf58d4f7ba/users/9b59f3c686c04e08bb7115640fbcdeb8/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.127670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902385f7-1a35-46e3-96ef-a1c20bb55f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa0885d35c84ca3ae116dcf58d4f7ba, user_id=9b59f3c686c04e08bb7115640fbcdeb8, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.130344 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902385f7-1a35-46e3-96ef-a1c20bb55f96 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.132588 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:54.148061 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1217/2439] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:54 2022] PUT /identity/v3/projects/3fa0885d35c84ca3ae116dcf58d4f7ba/users/9b59f3c686c04e08bb7115640fbcdeb8/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:54.160772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0c950-c6d3-48d4-9285-f3347e5b58c9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.161173 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0c950-c6d3-48d4-9285-f3347e5b58c9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.161471 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0c950-c6d3-48d4-9285-f3347e5b58c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.169494 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:54.193930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e1b0c950-c6d3-48d4-9285-f3347e5b58c9 None None] MFA Rules not processed for user `9b59f3c686c04e08bb7115640fbcdeb8`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:54.195228 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e1b0c950-c6d3-48d4-9285-f3347e5b58c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:54.264369 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1223/2440] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:54.269235 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902385f7-1a35-46e3-96ef-a1c20bb55f96 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.334196 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902385f7-1a35-46e3-96ef-a1c20bb55f96 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.339281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2a158-a095-4357-af13-1b6718521702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.339880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2a158-a095-4357-af13-1b6718521702 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.340164 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2a158-a095-4357-af13-1b6718521702 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.340443 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2a158-a095-4357-af13-1b6718521702 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.341077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2a158-a095-4357-af13-1b6718521702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.343043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2a158-a095-4357-af13-1b6718521702 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.372517 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1218/2441] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:54 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:54.377190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0c950-c6d3-48d4-9285-f3347e5b58c9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.389246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0c950-c6d3-48d4-9285-f3347e5b58c9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.451301 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264baf6e-b01f-40c3-b06a-30b204bba750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.451921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264baf6e-b01f-40c3-b06a-30b204bba750 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.452239 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264baf6e-b01f-40c3-b06a-30b204bba750 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.452509 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264baf6e-b01f-40c3-b06a-30b204bba750 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.453166 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264baf6e-b01f-40c3-b06a-30b204bba750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.455086 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264baf6e-b01f-40c3-b06a-30b204bba750 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.480472 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1224/2442] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:54 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:54.481743 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2a158-a095-4357-af13-1b6718521702 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.492537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2a158-a095-4357-af13-1b6718521702 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.497625 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f53e-a091-402e-a1f8-2a0a6795bc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.498337 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f53e-a091-402e-a1f8-2a0a6795bc56 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.498663 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f53e-a091-402e-a1f8-2a0a6795bc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.499052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f53e-a091-402e-a1f8-2a0a6795bc56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.499683 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836f53e-a091-402e-a1f8-2a0a6795bc56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.501355 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836f53e-a091-402e-a1f8-2a0a6795bc56 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.512427 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1219/2443] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:56:54.517041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264baf6e-b01f-40c3-b06a-30b204bba750 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.527292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264baf6e-b01f-40c3-b06a-30b204bba750 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.532115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3eddd-926a-41d1-b11e-0adc4616de02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.532762 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3eddd-926a-41d1-b11e-0adc4616de02 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.533044 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3eddd-926a-41d1-b11e-0adc4616de02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.533326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3eddd-926a-41d1-b11e-0adc4616de02 admin admin] PATH_INFO: `/v3/projects/75965f6d7260497ea808617fa6716a41/users/ba65ff898a3b43f7a912ed899998e10d/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.548539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3eddd-926a-41d1-b11e-0adc4616de02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75965f6d7260497ea808617fa6716a41, user_id=ba65ff898a3b43f7a912ed899998e10d, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.551596 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3eddd-926a-41d1-b11e-0adc4616de02 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.566097 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1225/2444] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:54 2022] PUT /identity/v3/projects/75965f6d7260497ea808617fa6716a41/users/ba65ff898a3b43f7a912ed899998e10d/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:54.570193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f53e-a091-402e-a1f8-2a0a6795bc56 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.580742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f53e-a091-402e-a1f8-2a0a6795bc56 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.639391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d569c-3ed2-4693-a263-ae1ec6c2fc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.640039 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d569c-3ed2-4693-a263-ae1ec6c2fc06 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.640314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d569c-3ed2-4693-a263-ae1ec6c2fc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.640583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d569c-3ed2-4693-a263-ae1ec6c2fc06 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.641213 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d569c-3ed2-4693-a263-ae1ec6c2fc06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.642847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d569c-3ed2-4693-a263-ae1ec6c2fc06 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.652551 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1220/2445] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:54.657028 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3eddd-926a-41d1-b11e-0adc4616de02 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.667504 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3eddd-926a-41d1-b11e-0adc4616de02 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.672398 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57700c-923e-4842-b136-b12aea3c5ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.673046 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57700c-923e-4842-b136-b12aea3c5ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.673326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57700c-923e-4842-b136-b12aea3c5ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.673599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57700c-923e-4842-b136-b12aea3c5ef1 admin admin] PATH_INFO: `/v3/projects/75965f6d7260497ea808617fa6716a41/users/ba65ff898a3b43f7a912ed899998e10d/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.675937 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57700c-923e-4842-b136-b12aea3c5ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75965f6d7260497ea808617fa6716a41, user_id=ba65ff898a3b43f7a912ed899998e10d, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.678568 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57700c-923e-4842-b136-b12aea3c5ef1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.688588 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1226/2446] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:54 2022] PUT /identity/v3/projects/75965f6d7260497ea808617fa6716a41/users/ba65ff898a3b43f7a912ed899998e10d/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:54.700495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0d38a-c38c-4411-8e81-3bc2331a8346 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.700767 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0d38a-c38c-4411-8e81-3bc2331a8346 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.701045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0d38a-c38c-4411-8e81-3bc2331a8346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.730271 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e6c0d38a-c38c-4411-8e81-3bc2331a8346 None None] MFA Rules not processed for user `ba65ff898a3b43f7a912ed899998e10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:54.731516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e6c0d38a-c38c-4411-8e81-3bc2331a8346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:54.787648 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:54.809149 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1221/2447] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:54.824091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57700c-923e-4842-b136-b12aea3c5ef1 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:54.825630 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5f57700c-923e-4842-b136-b12aea3c5ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:54.875324 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:54.886575 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57700c-923e-4842-b136-b12aea3c5ef1 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:54.890519 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbc6fe-6d16-4706-b24f-19dbe6638cfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:54.891102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbc6fe-6d16-4706-b24f-19dbe6638cfa service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:54.891375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbc6fe-6d16-4706-b24f-19dbe6638cfa service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:54.891641 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbc6fe-6d16-4706-b24f-19dbe6638cfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:54.946411 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbc6fe-6d16-4706-b24f-19dbe6638cfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:54.947915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbc6fe-6d16-4706-b24f-19dbe6638cfa service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:54.956771 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1227/2448] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:56:55.896788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0d38a-c38c-4411-8e81-3bc2331a8346 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:55.898454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e6c0d38a-c38c-4411-8e81-3bc2331a8346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:55.928598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbc6fe-6d16-4706-b24f-19dbe6638cfa service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:55.930106 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5adbc6fe-6d16-4706-b24f-19dbe6638cfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:55.966847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0d38a-c38c-4411-8e81-3bc2331a8346 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:55.971709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:55.972349 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:55.972629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:55.973079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] PATH_INFO: `/v3/users/d2478b9d83f440bf95cb104f0417f7a7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:55.974043 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2478b9d83f440bf95cb104f0417f7a7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:55.975996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:55.986081 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbc6fe-6d16-4706-b24f-19dbe6638cfa service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:55.989731 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:55.990361 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:55.990633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:55.990928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:55.992683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:56.025040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2478b9d83f440bf95cb104f0417f7a7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.025689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2478b9d83f440bf95cb104f0417f7a7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.033030 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2478b9d83f440bf95cb104f0417f7a7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.040985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2478b9d83f440bf95cb104f0417f7a7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.053229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78b4b7ed-3f98-43e2-ae1e-18b5efadb10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2478b9d83f440bf95cb104f0417f7a7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.059189 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1222/2449] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:56:55 2022] DELETE /identity/v3/users/d2478b9d83f440bf95cb104f0417f7a7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:56:56.063866 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b56e2-de1c-4168-a61b-676011038855 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:56.064139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:56.064253 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b56e2-de1c-4168-a61b-676011038855 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:56.064530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b56e2-de1c-4168-a61b-676011038855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:56.065590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:56.075670 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1228/2450] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:56:56.084700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6d3b56e2-de1c-4168-a61b-676011038855 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:56.087470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6d3b56e2-de1c-4168-a61b-676011038855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:56.147671 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1223/2451] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:56.157471 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:56.159206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:56.171291 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22950327-e075-40c5-a082-a478ae5ee004 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:56.219304 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:56.224741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccfbf4-4b05-4703-a3a7-a851c50d4060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:56.225662 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccfbf4-4b05-4703-a3a7-a851c50d4060 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:56.226115 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccfbf4-4b05-4703-a3a7-a851c50d4060 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:56.226601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccfbf4-4b05-4703-a3a7-a851c50d4060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:56.236546 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baccfbf4-4b05-4703-a3a7-a851c50d4060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:56.237986 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baccfbf4-4b05-4703-a3a7-a851c50d4060 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:56.248214 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1229/2452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:56.355577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b56e2-de1c-4168-a61b-676011038855 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:56.357007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6d3b56e2-de1c-4168-a61b-676011038855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:56.368518 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:56.445009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b56e2-de1c-4168-a61b-676011038855 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:56.450369 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcf603-f402-49f6-9e0b-bdeb57c6a0e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:56.451156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcf603-f402-49f6-9e0b-bdeb57c6a0e7 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:56.451596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcf603-f402-49f6-9e0b-bdeb57c6a0e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:56.451953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcf603-f402-49f6-9e0b-bdeb57c6a0e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:56.453813 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-88dcf603-f402-49f6-9e0b-bdeb57c6a0e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:56.530991 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccfbf4-4b05-4703-a3a7-a851c50d4060 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:56.533091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-baccfbf4-4b05-4703-a3a7-a851c50d4060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:56.538008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcf603-f402-49f6-9e0b-bdeb57c6a0e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:56.540922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcf603-f402-49f6-9e0b-bdeb57c6a0e7 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:56.547328 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccfbf4-4b05-4703-a3a7-a851c50d4060 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:56.553119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:56.553867 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:56.554298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:56.554560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] PATH_INFO: `/v3/projects/200fe485e7ce478cbd03c3fb62b24d6f` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:56.555943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200fe485e7ce478cbd03c3fb62b24d6f)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:56.558318 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:56.560283 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1224/2453] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:56.632512 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200fe485e7ce478cbd03c3fb62b24d6f', 'request_id': 'req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.639989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200fe485e7ce478cbd03c3fb62b24d6f', 'request_id': 'req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.648490 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200fe485e7ce478cbd03c3fb62b24d6f', 'request_id': 'req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.665755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] The token cache is being invalidate because project 200fe485e7ce478cbd03c3fb62b24d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:56:56.665755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.666574 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9d6874b6-a4a5-4c28-9c79-e63ffc223f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:56:56.668623 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1230/2454] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:56:56 2022] DELETE /identity/v3/projects/200fe485e7ce478cbd03c3fb62b24d6f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:56:56.674417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771503af-0533-49ff-9227-8c5d232362be None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:56.674696 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771503af-0533-49ff-9227-8c5d232362be None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:56.675712 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771503af-0533-49ff-9227-8c5d232362be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:56.702417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-771503af-0533-49ff-9227-8c5d232362be None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:56.704806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-771503af-0533-49ff-9227-8c5d232362be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:56.774176 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1225/2455] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:56.782498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0495cb-e8a7-43d2-9383-289d819014d2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:56.782908 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0495cb-e8a7-43d2-9383-289d819014d2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:56.783258 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0495cb-e8a7-43d2-9383-289d819014d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:56.805393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3b0495cb-e8a7-43d2-9383-289d819014d2 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:56.806826 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3b0495cb-e8a7-43d2-9383-289d819014d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:56.816596 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1231/2456] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:56.821781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771503af-0533-49ff-9227-8c5d232362be None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:56.831945 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771503af-0533-49ff-9227-8c5d232362be None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:56.836667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8b28f-f96b-4d7d-bff9-560df70ebdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:56.837375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8b28f-f96b-4d7d-bff9-560df70ebdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:56.837713 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8b28f-f96b-4d7d-bff9-560df70ebdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:56.838046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8b28f-f96b-4d7d-bff9-560df70ebdc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:56.838692 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8b28f-f96b-4d7d-bff9-560df70ebdc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:56:56.839019 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8b28f-f96b-4d7d-bff9-560df70ebdc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:56.840609 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8b28f-f96b-4d7d-bff9-560df70ebdc3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:56.854753 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1226/2457] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:56:56.857144 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0495cb-e8a7-43d2-9383-289d819014d2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:56.866565 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0495cb-e8a7-43d2-9383-289d819014d2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:56.872978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b1e38-124e-4bcf-9c8b-e67d8e2123a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:56.873732 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b1e38-124e-4bcf-9c8b-e67d8e2123a6 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:56.874004 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b1e38-124e-4bcf-9c8b-e67d8e2123a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:56.874813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b1e38-124e-4bcf-9c8b-e67d8e2123a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:56.874996 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b1e38-124e-4bcf-9c8b-e67d8e2123a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:56.877102 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b1e38-124e-4bcf-9c8b-e67d8e2123a6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:56.908526 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1232/2458] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:56:56 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:56.909749 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8b28f-f96b-4d7d-bff9-560df70ebdc3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:56.920379 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8b28f-f96b-4d7d-bff9-560df70ebdc3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:56.964551 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:56:56.981313 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35955cf-7a96-4349-bd05-2139c04d88e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:56.982083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35955cf-7a96-4349-bd05-2139c04d88e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:56.982377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35955cf-7a96-4349-bd05-2139c04d88e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:56.982654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35955cf-7a96-4349-bd05-2139c04d88e8 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:56.983342 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35955cf-7a96-4349-bd05-2139c04d88e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:56.985271 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35955cf-7a96-4349-bd05-2139c04d88e8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:57.006651 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1227/2459] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:56:56 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:56:57.010866 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b1e38-124e-4bcf-9c8b-e67d8e2123a6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:57.021980 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b1e38-124e-4bcf-9c8b-e67d8e2123a6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:57.027555 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed21a2b-790b-4553-a973-31980bf7ceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:57.028168 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed21a2b-790b-4553-a973-31980bf7ceb3 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:57.028478 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed21a2b-790b-4553-a973-31980bf7ceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:57.028744 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed21a2b-790b-4553-a973-31980bf7ceb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:57.029387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed21a2b-790b-4553-a973-31980bf7ceb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:57.031196 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed21a2b-790b-4553-a973-31980bf7ceb3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:57.041844 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1233/2460] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:57.046113 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35955cf-7a96-4349-bd05-2139c04d88e8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:57.056782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35955cf-7a96-4349-bd05-2139c04d88e8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:57.061925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b25a0-5dd4-4726-b6fa-aebc32ad635a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:57.062577 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b25a0-5dd4-4726-b6fa-aebc32ad635a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:57.062898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b25a0-5dd4-4726-b6fa-aebc32ad635a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:57.063173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b25a0-5dd4-4726-b6fa-aebc32ad635a admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:57.063846 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b25a0-5dd4-4726-b6fa-aebc32ad635a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:57.065546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b25a0-5dd4-4726-b6fa-aebc32ad635a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:57.077870 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1228/2461] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:56:57.082076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed21a2b-790b-4553-a973-31980bf7ceb3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:57.092136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed21a2b-790b-4553-a973-31980bf7ceb3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:57.096886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f734df7-886e-4eed-8d75-c8edc4b98f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:57.097543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f734df7-886e-4eed-8d75-c8edc4b98f72 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:57.097821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f734df7-886e-4eed-8d75-c8edc4b98f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:57.098091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f734df7-886e-4eed-8d75-c8edc4b98f72 admin admin] PATH_INFO: `/v3/projects/77df0bbca3c740b789fe7e0ea62f9e3a/users/8926edfa5bbd48ae91bca6e01dd0e642/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:57.114598 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f734df7-886e-4eed-8d75-c8edc4b98f72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77df0bbca3c740b789fe7e0ea62f9e3a, user_id=8926edfa5bbd48ae91bca6e01dd0e642, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:57.117880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f734df7-886e-4eed-8d75-c8edc4b98f72 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:57.131396 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1234/2462] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:56:57 2022] PUT /identity/v3/projects/77df0bbca3c740b789fe7e0ea62f9e3a/users/8926edfa5bbd48ae91bca6e01dd0e642/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:56:57.143062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20e4d4-a2c0-4074-a15c-1625429f21d4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:57.143351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20e4d4-a2c0-4074-a15c-1625429f21d4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:57.143622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20e4d4-a2c0-4074-a15c-1625429f21d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:57.175358 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0f20e4d4-a2c0-4074-a15c-1625429f21d4 None None] MFA Rules not processed for user `8926edfa5bbd48ae91bca6e01dd0e642`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:57.176582 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0f20e4d4-a2c0-4074-a15c-1625429f21d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:57.240000 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1229/2463] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:56:57.251152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f734df7-886e-4eed-8d75-c8edc4b98f72 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:56:57.253084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4f734df7-886e-4eed-8d75-c8edc4b98f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:57.315520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f734df7-886e-4eed-8d75-c8edc4b98f72 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:56:57.319829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f976cc1-9502-435d-b389-db9be31ebe95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:56:57.320835 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f976cc1-9502-435d-b389-db9be31ebe95 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:57.321109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f976cc1-9502-435d-b389-db9be31ebe95 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:57.321377 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f976cc1-9502-435d-b389-db9be31ebe95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:57.331198 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f976cc1-9502-435d-b389-db9be31ebe95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:56:57.332700 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f976cc1-9502-435d-b389-db9be31ebe95 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:56:57.342886 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1235/2464] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:56:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:56:58.729682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85b60f-1646-493a-8107-41b57e04f9c0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:56:58.730276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85b60f-1646-493a-8107-41b57e04f9c0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:56:58.730276 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85b60f-1646-493a-8107-41b57e04f9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:56:58.751634 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-aa85b60f-1646-493a-8107-41b57e04f9c0 None None] MFA Rules not processed for user `8926edfa5bbd48ae91bca6e01dd0e642`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:56:58.753114 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-aa85b60f-1646-493a-8107-41b57e04f9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:56:58.766632 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1230/2465] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:56:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:00.451589 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f976cc1-9502-435d-b389-db9be31ebe95 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:00.453583 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7f976cc1-9502-435d-b389-db9be31ebe95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:00.468827 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:00.523249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f976cc1-9502-435d-b389-db9be31ebe95 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:00.527718 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:00.528356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:00.528682 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:00.528903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] PATH_INFO: `/v3/users/9b59f3c686c04e08bb7115640fbcdeb8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:00.530232 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b59f3c686c04e08bb7115640fbcdeb8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:00.532226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:00.580286 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b59f3c686c04e08bb7115640fbcdeb8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:00.587243 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b59f3c686c04e08bb7115640fbcdeb8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:00.587914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b59f3c686c04e08bb7115640fbcdeb8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:00.596566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b59f3c686c04e08bb7115640fbcdeb8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:00.608323 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b59f3c686c04e08bb7115640fbcdeb8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:00.614439 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1236/2466] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:57:00 2022] DELETE /identity/v3/users/9b59f3c686c04e08bb7115640fbcdeb8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:00.619154 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5a2c2-bb20-4277-a25a-cc9f75304d80 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:00.619631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5a2c2-bb20-4277-a25a-cc9f75304d80 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:00.619745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5a2c2-bb20-4277-a25a-cc9f75304d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:00.638847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-36a5a2c2-bb20-4277-a25a-cc9f75304d80 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:00.640071 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-36a5a2c2-bb20-4277-a25a-cc9f75304d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:00.700299 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1231/2467] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:00.710301 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:00.711757 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:00.768755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f371eff-038f-4ec1-84dc-d2caa93deccd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:00.772567 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e587bc0-3121-4014-a801-c50c5560fed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:00.773187 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e587bc0-3121-4014-a801-c50c5560fed5 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:00.773467 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e587bc0-3121-4014-a801-c50c5560fed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:00.773739 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e587bc0-3121-4014-a801-c50c5560fed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:00.783216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e587bc0-3121-4014-a801-c50c5560fed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:00.784657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e587bc0-3121-4014-a801-c50c5560fed5 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:00.794732 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1237/2468] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:01.064074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5a2c2-bb20-4277-a25a-cc9f75304d80 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:01.065553 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-36a5a2c2-bb20-4277-a25a-cc9f75304d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:01.077636 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5a2c2-bb20-4277-a25a-cc9f75304d80 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:01.082277 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:01.083053 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:01.083560 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:01.083902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] PATH_INFO: `/v3/projects/3fa0885d35c84ca3ae116dcf58d4f7ba` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:01.085255 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa0885d35c84ca3ae116dcf58d4f7ba)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:01.087525 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:01.133226 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:01.137692 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa0885d35c84ca3ae116dcf58d4f7ba', 'request_id': 'req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.145044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa0885d35c84ca3ae116dcf58d4f7ba', 'request_id': 'req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.151495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa0885d35c84ca3ae116dcf58d4f7ba', 'request_id': 'req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.165985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] The token cache is being invalidate because project 3fa0885d35c84ca3ae116dcf58d4f7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:57:01.166270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.166912 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.168442 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1232/2469] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:57:01 2022] DELETE /identity/v3/projects/3fa0885d35c84ca3ae116dcf58d4f7ba => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:01.172558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e587bc0-3121-4014-a801-c50c5560fed5 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:01.191788 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e587bc0-3121-4014-a801-c50c5560fed5 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:01.228043 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:01.255240 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:01.255899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:01.256158 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:01.256433 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] PATH_INFO: `/v3/users/ba65ff898a3b43f7a912ed899998e10d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:01.257873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba65ff898a3b43f7a912ed899998e10d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:01.263282 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:01.324802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba65ff898a3b43f7a912ed899998e10d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.332406 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba65ff898a3b43f7a912ed899998e10d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.333108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba65ff898a3b43f7a912ed899998e10d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.344012 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba65ff898a3b43f7a912ed899998e10d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.357779 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ec1e71e9-a57e-48be-8654-6a93924f3768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba65ff898a3b43f7a912ed899998e10d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:01.365228 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1238/2470] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:57:01 2022] DELETE /identity/v3/users/ba65ff898a3b43f7a912ed899998e10d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:01.940479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:01.942641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:01.959678 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1104d9-1d40-4afa-8144-5e5bd8fd494f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.018785 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.019478 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.019769 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.020035 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] PATH_INFO: `/v3/projects/75965f6d7260497ea808617fa6716a41` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.021183 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75965f6d7260497ea808617fa6716a41)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.023132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.071587 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75965f6d7260497ea808617fa6716a41', 'request_id': 'req-78461e40-0452-4b4b-a29d-11050739f7c1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:02.077951 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75965f6d7260497ea808617fa6716a41', 'request_id': 'req-78461e40-0452-4b4b-a29d-11050739f7c1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:02.085046 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75965f6d7260497ea808617fa6716a41', 'request_id': 'req-78461e40-0452-4b4b-a29d-11050739f7c1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:02.105238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] The token cache is being invalidate because project 75965f6d7260497ea808617fa6716a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:57:02.105758 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:02.106419 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-78461e40-0452-4b4b-a29d-11050739f7c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:02.107972 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1233/2471] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:57:01 2022] DELETE /identity/v3/projects/75965f6d7260497ea808617fa6716a41 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:02.114292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673f8fe-7f53-4e6e-9d3e-a138b3780136 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.114804 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673f8fe-7f53-4e6e-9d3e-a138b3780136 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.114959 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673f8fe-7f53-4e6e-9d3e-a138b3780136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.135069 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6673f8fe-7f53-4e6e-9d3e-a138b3780136 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:02.136199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6673f8fe-7f53-4e6e-9d3e-a138b3780136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:02.201355 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1239/2472] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:02.209674 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebad22-014e-43fe-87db-880b173b7faa None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.209948 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebad22-014e-43fe-87db-880b173b7faa None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.210254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebad22-014e-43fe-87db-880b173b7faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.233795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4aebad22-014e-43fe-87db-880b173b7faa None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:02.235456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4aebad22-014e-43fe-87db-880b173b7faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:02.245221 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1234/2473] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:57:02.250011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673f8fe-7f53-4e6e-9d3e-a138b3780136 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.260458 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673f8fe-7f53-4e6e-9d3e-a138b3780136 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.265190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042469ab-b065-45d7-982b-d6f13af6c6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.265905 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042469ab-b065-45d7-982b-d6f13af6c6de admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.266250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042469ab-b065-45d7-982b-d6f13af6c6de admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.266589 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042469ab-b065-45d7-982b-d6f13af6c6de admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.267347 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042469ab-b065-45d7-982b-d6f13af6c6de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:57:02.267715 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042469ab-b065-45d7-982b-d6f13af6c6de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.269571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042469ab-b065-45d7-982b-d6f13af6c6de admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.283445 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1240/2474] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:57:02.287779 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebad22-014e-43fe-87db-880b173b7faa None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.297215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebad22-014e-43fe-87db-880b173b7faa None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.302362 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66aa88-e3b7-42d9-82c8-fe26a566dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.303039 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66aa88-e3b7-42d9-82c8-fe26a566dcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.303318 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66aa88-e3b7-42d9-82c8-fe26a566dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.303594 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66aa88-e3b7-42d9-82c8-fe26a566dcf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.304244 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66aa88-e3b7-42d9-82c8-fe26a566dcf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.306138 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66aa88-e3b7-42d9-82c8-fe26a566dcf9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.334020 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1235/2475] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:57:02 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:02.338353 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042469ab-b065-45d7-982b-d6f13af6c6de admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.348771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042469ab-b065-45d7-982b-d6f13af6c6de admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.389736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:02.409298 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d953726-64a1-4345-b3cd-abf08142bf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.409917 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d953726-64a1-4345-b3cd-abf08142bf4c admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.410212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d953726-64a1-4345-b3cd-abf08142bf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.410481 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d953726-64a1-4345-b3cd-abf08142bf4c admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.411142 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d953726-64a1-4345-b3cd-abf08142bf4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.413027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d953726-64a1-4345-b3cd-abf08142bf4c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.435153 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1241/2476] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:57:02 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:02.439455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66aa88-e3b7-42d9-82c8-fe26a566dcf9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.449766 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66aa88-e3b7-42d9-82c8-fe26a566dcf9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.454467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d142ec-38d3-44c1-bd5e-4ea3d69fec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.455094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d142ec-38d3-44c1-bd5e-4ea3d69fec7f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.455373 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d142ec-38d3-44c1-bd5e-4ea3d69fec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.455646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d142ec-38d3-44c1-bd5e-4ea3d69fec7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.456320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d142ec-38d3-44c1-bd5e-4ea3d69fec7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.457914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d142ec-38d3-44c1-bd5e-4ea3d69fec7f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.470888 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1236/2477] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:57:02.474970 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d953726-64a1-4345-b3cd-abf08142bf4c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.485132 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d953726-64a1-4345-b3cd-abf08142bf4c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.490422 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac09fd-3fad-4ab4-b138-9b79f8550868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.491059 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac09fd-3fad-4ab4-b138-9b79f8550868 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.491342 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac09fd-3fad-4ab4-b138-9b79f8550868 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.491609 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac09fd-3fad-4ab4-b138-9b79f8550868 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.492250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac09fd-3fad-4ab4-b138-9b79f8550868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.493843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac09fd-3fad-4ab4-b138-9b79f8550868 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.505096 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1242/2478] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:02.509581 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d142ec-38d3-44c1-bd5e-4ea3d69fec7f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.520086 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d142ec-38d3-44c1-bd5e-4ea3d69fec7f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.524931 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e37cf3-3a5d-4ef1-ae3e-94c101c98932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.525538 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e37cf3-3a5d-4ef1-ae3e-94c101c98932 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.525842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e37cf3-3a5d-4ef1-ae3e-94c101c98932 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.526116 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e37cf3-3a5d-4ef1-ae3e-94c101c98932 admin admin] PATH_INFO: `/v3/projects/40ea2ca610f24d14a016d90377dd90ed/users/dd9fd16ee6c542cabac2a557c78c8e7e/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.541431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e37cf3-3a5d-4ef1-ae3e-94c101c98932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ea2ca610f24d14a016d90377dd90ed, user_id=dd9fd16ee6c542cabac2a557c78c8e7e, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.544121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e37cf3-3a5d-4ef1-ae3e-94c101c98932 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.554527 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:02.561617 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1237/2479] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:02 2022] PUT /identity/v3/projects/40ea2ca610f24d14a016d90377dd90ed/users/dd9fd16ee6c542cabac2a557c78c8e7e/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:57:02.573459 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b25b7-6b02-43da-8668-7547d489c011 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.573742 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b25b7-6b02-43da-8668-7547d489c011 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.574016 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b25b7-6b02-43da-8668-7547d489c011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.603206 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e29b25b7-6b02-43da-8668-7547d489c011 None None] MFA Rules not processed for user `dd9fd16ee6c542cabac2a557c78c8e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:02.604415 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e29b25b7-6b02-43da-8668-7547d489c011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:02.671687 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1243/2480] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:02.676617 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e37cf3-3a5d-4ef1-ae3e-94c101c98932 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.740642 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e37cf3-3a5d-4ef1-ae3e-94c101c98932 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.745328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152f503-ff3d-4fc7-b68b-e4e40a980de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.745925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152f503-ff3d-4fc7-b68b-e4e40a980de1 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.746202 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152f503-ff3d-4fc7-b68b-e4e40a980de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.746472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152f503-ff3d-4fc7-b68b-e4e40a980de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.747153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152f503-ff3d-4fc7-b68b-e4e40a980de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.749094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152f503-ff3d-4fc7-b68b-e4e40a980de1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.778930 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1238/2481] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:57:02 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:02.783387 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b25b7-6b02-43da-8668-7547d489c011 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.794886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b25b7-6b02-43da-8668-7547d489c011 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.851450 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5382ad6-aca1-48f9-8fff-d4789c69095f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.852356 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5382ad6-aca1-48f9-8fff-d4789c69095f admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.852629 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5382ad6-aca1-48f9-8fff-d4789c69095f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.852897 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5382ad6-aca1-48f9-8fff-d4789c69095f admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.853529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5382ad6-aca1-48f9-8fff-d4789c69095f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.855467 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5382ad6-aca1-48f9-8fff-d4789c69095f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.878869 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1244/2482] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:57:02 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:02.883018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152f503-ff3d-4fc7-b68b-e4e40a980de1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.893810 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152f503-ff3d-4fc7-b68b-e4e40a980de1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.898602 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409ce00-ffef-4748-8e20-cc4c05fdbcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.899284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409ce00-ffef-4748-8e20-cc4c05fdbcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.899596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409ce00-ffef-4748-8e20-cc4c05fdbcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.899849 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409ce00-ffef-4748-8e20-cc4c05fdbcf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.900517 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409ce00-ffef-4748-8e20-cc4c05fdbcf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.902106 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409ce00-ffef-4748-8e20-cc4c05fdbcf6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.916371 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1239/2483] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:02.917633 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5382ad6-aca1-48f9-8fff-d4789c69095f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.928901 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5382ad6-aca1-48f9-8fff-d4789c69095f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:02.934103 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f587af-bf27-49b8-81ba-d97ad0cbc580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:02.934799 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f587af-bf27-49b8-81ba-d97ad0cbc580 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:02.935079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f587af-bf27-49b8-81ba-d97ad0cbc580 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:02.935348 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f587af-bf27-49b8-81ba-d97ad0cbc580 admin admin] PATH_INFO: `/v3/projects/eb3c183c0f1b4230853472a803677eb1/users/6f47e4c013da4426bdf68ffaf382c407/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:02.951042 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f587af-bf27-49b8-81ba-d97ad0cbc580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3c183c0f1b4230853472a803677eb1, user_id=6f47e4c013da4426bdf68ffaf382c407, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:02.953704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f587af-bf27-49b8-81ba-d97ad0cbc580 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:02.970380 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1245/2484] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:02 2022] PUT /identity/v3/projects/eb3c183c0f1b4230853472a803677eb1/users/6f47e4c013da4426bdf68ffaf382c407/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:02.974380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409ce00-ffef-4748-8e20-cc4c05fdbcf6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:02.984454 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409ce00-ffef-4748-8e20-cc4c05fdbcf6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:03.039739 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfa0a8-63af-49e2-8031-f6c546b03f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:03.040345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfa0a8-63af-49e2-8031-f6c546b03f09 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:03.040618 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfa0a8-63af-49e2-8031-f6c546b03f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:03.040884 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfa0a8-63af-49e2-8031-f6c546b03f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:03.041516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfa0a8-63af-49e2-8031-f6c546b03f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:03.043132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfa0a8-63af-49e2-8031-f6c546b03f09 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:03.053287 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1240/2485] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:03.057618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f587af-bf27-49b8-81ba-d97ad0cbc580 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:03.068295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f587af-bf27-49b8-81ba-d97ad0cbc580 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:03.073242 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155d990-418e-4cfb-8b04-208956af926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:03.073880 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155d990-418e-4cfb-8b04-208956af926a admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:03.074156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155d990-418e-4cfb-8b04-208956af926a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:03.074422 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155d990-418e-4cfb-8b04-208956af926a admin admin] PATH_INFO: `/v3/projects/eb3c183c0f1b4230853472a803677eb1/users/6f47e4c013da4426bdf68ffaf382c407/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:03.076531 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155d990-418e-4cfb-8b04-208956af926a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3c183c0f1b4230853472a803677eb1, user_id=6f47e4c013da4426bdf68ffaf382c407, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:03.079182 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155d990-418e-4cfb-8b04-208956af926a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:03.089434 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1246/2486] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:03 2022] PUT /identity/v3/projects/eb3c183c0f1b4230853472a803677eb1/users/6f47e4c013da4426bdf68ffaf382c407/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:03.099776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744bca8-fc69-45e0-bec1-c95a46a3bf21 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:03.100111 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744bca8-fc69-45e0-bec1-c95a46a3bf21 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:03.100364 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744bca8-fc69-45e0-bec1-c95a46a3bf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:03.128631 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7744bca8-fc69-45e0-bec1-c95a46a3bf21 None None] MFA Rules not processed for user `6f47e4c013da4426bdf68ffaf382c407`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:03.129806 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7744bca8-fc69-45e0-bec1-c95a46a3bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:03.190842 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:03.203878 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1241/2487] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:03.215418 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155d990-418e-4cfb-8b04-208956af926a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:03.217337 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f155d990-418e-4cfb-8b04-208956af926a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:03.223041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:03.276834 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155d990-418e-4cfb-8b04-208956af926a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:03.281067 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5761d96-b865-480b-be7b-eabe334b86a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:03.281683 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5761d96-b865-480b-be7b-eabe334b86a6 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:03.281956 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5761d96-b865-480b-be7b-eabe334b86a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:03.282235 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5761d96-b865-480b-be7b-eabe334b86a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:03.292363 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5761d96-b865-480b-be7b-eabe334b86a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:03.293854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5761d96-b865-480b-be7b-eabe334b86a6 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:03.303834 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1247/2488] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:03.370640 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744bca8-fc69-45e0-bec1-c95a46a3bf21 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:03.434675 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744bca8-fc69-45e0-bec1-c95a46a3bf21 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:03.439696 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:03.440317 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:03.440597 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:03.440929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] PATH_INFO: `/v3/users/dd9fd16ee6c542cabac2a557c78c8e7e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:03.442052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9fd16ee6c542cabac2a557c78c8e7e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:03.444000 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:03.492647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9fd16ee6c542cabac2a557c78c8e7e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:03.493281 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9fd16ee6c542cabac2a557c78c8e7e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:03.500175 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9fd16ee6c542cabac2a557c78c8e7e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:03.509361 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9fd16ee6c542cabac2a557c78c8e7e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:03.522550 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9fd16ee6c542cabac2a557c78c8e7e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:03.528760 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1242/2489] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:57:03 2022] DELETE /identity/v3/users/dd9fd16ee6c542cabac2a557c78c8e7e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:03.534785 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1645ad-601b-4c76-b4b8-ab60515640a4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:03.535085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1645ad-601b-4c76-b4b8-ab60515640a4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:03.535355 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1645ad-601b-4c76-b4b8-ab60515640a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:03.555774 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8b1645ad-601b-4c76-b4b8-ab60515640a4 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:03.556965 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8b1645ad-601b-4c76-b4b8-ab60515640a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:03.622707 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1248/2490] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:03.632954 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:03.634616 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:03.694912 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbbc4-3bdc-4eb2-86fb-9349ef2f327e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:03.698925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873c3b3-40d3-4c82-87bc-3f2270f29bfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:03.699524 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873c3b3-40d3-4c82-87bc-3f2270f29bfa service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:03.699805 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873c3b3-40d3-4c82-87bc-3f2270f29bfa service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:03.700073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873c3b3-40d3-4c82-87bc-3f2270f29bfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:03.709484 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873c3b3-40d3-4c82-87bc-3f2270f29bfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:03.710949 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873c3b3-40d3-4c82-87bc-3f2270f29bfa service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:03.720449 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1243/2491] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:03.990079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1645ad-601b-4c76-b4b8-ab60515640a4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:03.991704 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8b1645ad-601b-4c76-b4b8-ab60515640a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:04.006852 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1645ad-601b-4c76-b4b8-ab60515640a4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:04.011837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:04.013113 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:04.013419 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:04.014608 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] PATH_INFO: `/v3/projects/40ea2ca610f24d14a016d90377dd90ed` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:04.015982 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ea2ca610f24d14a016d90377dd90ed)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:04.017914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:04.069569 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ea2ca610f24d14a016d90377dd90ed', 'request_id': 'req-dc18c9ab-f071-4c53-a4bc-f10115aca719'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.075047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ea2ca610f24d14a016d90377dd90ed', 'request_id': 'req-dc18c9ab-f071-4c53-a4bc-f10115aca719'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.081560 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ea2ca610f24d14a016d90377dd90ed', 'request_id': 'req-dc18c9ab-f071-4c53-a4bc-f10115aca719'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.096060 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] The token cache is being invalidate because project 40ea2ca610f24d14a016d90377dd90ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:57:04.096350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.096974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.098476 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1249/2492] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:57:03 2022] DELETE /identity/v3/projects/40ea2ca610f24d14a016d90377dd90ed => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:04.109285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873c3b3-40d3-4c82-87bc-3f2270f29bfa service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:04.120179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873c3b3-40d3-4c82-87bc-3f2270f29bfa service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:04.132815 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:04.182959 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:04.183592 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:04.183872 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:04.184151 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] PATH_INFO: `/v3/users/6f47e4c013da4426bdf68ffaf382c407` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:04.185274 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f47e4c013da4426bdf68ffaf382c407)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:04.187233 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:04.232600 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f47e4c013da4426bdf68ffaf382c407'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.233158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f47e4c013da4426bdf68ffaf382c407'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.240270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f47e4c013da4426bdf68ffaf382c407'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.249555 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f47e4c013da4426bdf68ffaf382c407'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.262004 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-43f07aea-dc73-4df1-9cbf-5a686d9c922b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f47e4c013da4426bdf68ffaf382c407'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.268640 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1244/2493] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:57:04 2022] DELETE /identity/v3/users/6f47e4c013da4426bdf68ffaf382c407 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:04.545383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:04.546929 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:04.561430 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18c9ab-f071-4c53-a4bc-f10115aca719 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:04.591915 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:04.624938 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:04.625571 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:04.625849 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:04.626119 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] PATH_INFO: `/v3/projects/eb3c183c0f1b4230853472a803677eb1` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:04.627295 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3c183c0f1b4230853472a803677eb1)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:04.629254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:04.675463 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3c183c0f1b4230853472a803677eb1', 'request_id': 'req-4702dae5-b7c1-4690-a916-ecd362a8d05d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.681073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3c183c0f1b4230853472a803677eb1', 'request_id': 'req-4702dae5-b7c1-4690-a916-ecd362a8d05d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.687211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3c183c0f1b4230853472a803677eb1', 'request_id': 'req-4702dae5-b7c1-4690-a916-ecd362a8d05d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.701431 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] The token cache is being invalidate because project eb3c183c0f1b4230853472a803677eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:57:04.701722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.702346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4702dae5-b7c1-4690-a916-ecd362a8d05d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:04.704101 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1250/2494] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:57:04 2022] DELETE /identity/v3/projects/eb3c183c0f1b4230853472a803677eb1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:04.709132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d74dd-eb58-40ba-b054-6d4dcf6ba3e3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:04.709485 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d74dd-eb58-40ba-b054-6d4dcf6ba3e3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:04.709757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d74dd-eb58-40ba-b054-6d4dcf6ba3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:04.729503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3a8d74dd-eb58-40ba-b054-6d4dcf6ba3e3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:04.730698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3a8d74dd-eb58-40ba-b054-6d4dcf6ba3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:04.789926 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1245/2495] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:04.797672 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340d286-d18c-4a28-9513-8f59675d8451 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:04.798050 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340d286-d18c-4a28-9513-8f59675d8451 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:04.798391 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340d286-d18c-4a28-9513-8f59675d8451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:04.819474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1340d286-d18c-4a28-9513-8f59675d8451 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:04.821409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1340d286-d18c-4a28-9513-8f59675d8451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:04.829847 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1251/2496] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:04.834215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d74dd-eb58-40ba-b054-6d4dcf6ba3e3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:04.843987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d74dd-eb58-40ba-b054-6d4dcf6ba3e3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:04.848397 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c315931-a918-40f1-bb7b-54eef7207ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:04.848992 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c315931-a918-40f1-bb7b-54eef7207ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:04.849268 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c315931-a918-40f1-bb7b-54eef7207ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:04.849573 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c315931-a918-40f1-bb7b-54eef7207ca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:04.850216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c315931-a918-40f1-bb7b-54eef7207ca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:57:04.850511 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c315931-a918-40f1-bb7b-54eef7207ca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:04.852139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c315931-a918-40f1-bb7b-54eef7207ca8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:04.863796 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1246/2497] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:57:04.868297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340d286-d18c-4a28-9513-8f59675d8451 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:04.877410 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340d286-d18c-4a28-9513-8f59675d8451 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:04.882290 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87a58f-9b26-426e-9413-da4eb6280406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:04.882951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87a58f-9b26-426e-9413-da4eb6280406 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:04.883233 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87a58f-9b26-426e-9413-da4eb6280406 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:04.883534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87a58f-9b26-426e-9413-da4eb6280406 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:04.884175 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87a58f-9b26-426e-9413-da4eb6280406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:04.886093 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87a58f-9b26-426e-9413-da4eb6280406 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:04.918168 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1252/2498] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:57:04 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:57:04.919654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c315931-a918-40f1-bb7b-54eef7207ca8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:04.930768 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c315931-a918-40f1-bb7b-54eef7207ca8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:04.951484 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:04.991151 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daed081-6794-40cf-beb1-9b9d5866a563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:04.991856 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daed081-6794-40cf-beb1-9b9d5866a563 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:04.992135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daed081-6794-40cf-beb1-9b9d5866a563 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:04.993003 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daed081-6794-40cf-beb1-9b9d5866a563 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:04.993658 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daed081-6794-40cf-beb1-9b9d5866a563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:04.995584 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daed081-6794-40cf-beb1-9b9d5866a563 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.021142 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1247/2499] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:57:04 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:57:05.023207 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87a58f-9b26-426e-9413-da4eb6280406 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.043021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87a58f-9b26-426e-9413-da4eb6280406 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.050165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ea11e-987b-4796-a088-97d716d7dc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.050873 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ea11e-987b-4796-a088-97d716d7dc38 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.051152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ea11e-987b-4796-a088-97d716d7dc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.051429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ea11e-987b-4796-a088-97d716d7dc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.052094 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ea11e-987b-4796-a088-97d716d7dc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.053885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ea11e-987b-4796-a088-97d716d7dc38 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.067751 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1253/2500] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:05.069118 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daed081-6794-40cf-beb1-9b9d5866a563 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.079764 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daed081-6794-40cf-beb1-9b9d5866a563 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.085033 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5781b-2035-493c-818f-078e340a53ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.085194 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5781b-2035-493c-818f-078e340a53ab admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.085331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5781b-2035-493c-818f-078e340a53ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.085600 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5781b-2035-493c-818f-078e340a53ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.086238 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5781b-2035-493c-818f-078e340a53ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.087896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5781b-2035-493c-818f-078e340a53ab admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.098583 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1248/2501] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:05.102951 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ea11e-987b-4796-a088-97d716d7dc38 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.112957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ea11e-987b-4796-a088-97d716d7dc38 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.117610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a76d51-225f-44df-ad7e-63b56b858de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.118256 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a76d51-225f-44df-ad7e-63b56b858de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.118537 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a76d51-225f-44df-ad7e-63b56b858de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.118928 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a76d51-225f-44df-ad7e-63b56b858de5 admin admin] PATH_INFO: `/v3/projects/243b1115c22b4a8ba4d94ccb5aa7d770/users/e91cf10a974b4f25977e34213d0bf206/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.134312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a76d51-225f-44df-ad7e-63b56b858de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243b1115c22b4a8ba4d94ccb5aa7d770, user_id=e91cf10a974b4f25977e34213d0bf206, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.136997 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a76d51-225f-44df-ad7e-63b56b858de5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.151373 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1254/2502] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:05 2022] PUT /identity/v3/projects/243b1115c22b4a8ba4d94ccb5aa7d770/users/e91cf10a974b4f25977e34213d0bf206/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:05.164147 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac27162-7789-4732-a01d-687876f38549 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.164711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac27162-7789-4732-a01d-687876f38549 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.164711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac27162-7789-4732-a01d-687876f38549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.194122 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4ac27162-7789-4732-a01d-687876f38549 None None] MFA Rules not processed for user `e91cf10a974b4f25977e34213d0bf206`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:05.195627 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4ac27162-7789-4732-a01d-687876f38549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:05.262183 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1249/2503] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:05.271779 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a76d51-225f-44df-ad7e-63b56b858de5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.336638 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:05.342136 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a76d51-225f-44df-ad7e-63b56b858de5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.348549 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0b000-1dc9-44c9-9181-5dfd139c252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.349199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0b000-1dc9-44c9-9181-5dfd139c252e admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.349632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0b000-1dc9-44c9-9181-5dfd139c252e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.350057 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0b000-1dc9-44c9-9181-5dfd139c252e admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.350899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0b000-1dc9-44c9-9181-5dfd139c252e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.352921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0b000-1dc9-44c9-9181-5dfd139c252e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.381122 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1255/2504] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:57:05 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:05.385414 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac27162-7789-4732-a01d-687876f38549 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.396808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac27162-7789-4732-a01d-687876f38549 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.454974 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa895f1-076e-4cf4-8fde-5ab82ed2caec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.455682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa895f1-076e-4cf4-8fde-5ab82ed2caec admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.455962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa895f1-076e-4cf4-8fde-5ab82ed2caec admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.456234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa895f1-076e-4cf4-8fde-5ab82ed2caec admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.456875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa895f1-076e-4cf4-8fde-5ab82ed2caec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.458795 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa895f1-076e-4cf4-8fde-5ab82ed2caec admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.482598 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1250/2505] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:57:05 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:57:05.486740 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0b000-1dc9-44c9-9181-5dfd139c252e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.496938 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0b000-1dc9-44c9-9181-5dfd139c252e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.501864 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8d06f-e2bb-402d-9239-ac140c535e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.502457 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8d06f-e2bb-402d-9239-ac140c535e7a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.502755 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8d06f-e2bb-402d-9239-ac140c535e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.503025 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8d06f-e2bb-402d-9239-ac140c535e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.503655 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8d06f-e2bb-402d-9239-ac140c535e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.505257 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8d06f-e2bb-402d-9239-ac140c535e7a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.515827 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1256/2506] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:05.520197 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa895f1-076e-4cf4-8fde-5ab82ed2caec admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.530367 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa895f1-076e-4cf4-8fde-5ab82ed2caec admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.535388 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d52a2-b810-46f9-a9ad-b83183f9e944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.536025 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d52a2-b810-46f9-a9ad-b83183f9e944 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.536297 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d52a2-b810-46f9-a9ad-b83183f9e944 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.536583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d52a2-b810-46f9-a9ad-b83183f9e944 admin admin] PATH_INFO: `/v3/projects/a1b3b1f26e134ed4a63e15242a08a1a1/users/1755e4d8862d4811a0909ddd6f26dbc0/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.552473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d52a2-b810-46f9-a9ad-b83183f9e944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b3b1f26e134ed4a63e15242a08a1a1, user_id=1755e4d8862d4811a0909ddd6f26dbc0, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.555129 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d52a2-b810-46f9-a9ad-b83183f9e944 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.558325 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:05.572333 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1251/2507] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:05 2022] PUT /identity/v3/projects/a1b3b1f26e134ed4a63e15242a08a1a1/users/1755e4d8862d4811a0909ddd6f26dbc0/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:05.576281 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8d06f-e2bb-402d-9239-ac140c535e7a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.587325 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8d06f-e2bb-402d-9239-ac140c535e7a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.646253 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689edda-19e1-4f13-88ef-640453fcc980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.646981 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689edda-19e1-4f13-88ef-640453fcc980 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.647267 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689edda-19e1-4f13-88ef-640453fcc980 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.647534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689edda-19e1-4f13-88ef-640453fcc980 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.648175 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689edda-19e1-4f13-88ef-640453fcc980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.649795 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689edda-19e1-4f13-88ef-640453fcc980 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.659835 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1257/2508] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:05.663906 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d52a2-b810-46f9-a9ad-b83183f9e944 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.674840 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d52a2-b810-46f9-a9ad-b83183f9e944 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.679941 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63951aa7-184d-464a-bfd2-3f606d56e502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.680580 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63951aa7-184d-464a-bfd2-3f606d56e502 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.680900 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63951aa7-184d-464a-bfd2-3f606d56e502 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.681175 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63951aa7-184d-464a-bfd2-3f606d56e502 admin admin] PATH_INFO: `/v3/projects/a1b3b1f26e134ed4a63e15242a08a1a1/users/1755e4d8862d4811a0909ddd6f26dbc0/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.683237 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63951aa7-184d-464a-bfd2-3f606d56e502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b3b1f26e134ed4a63e15242a08a1a1, user_id=1755e4d8862d4811a0909ddd6f26dbc0, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.685843 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63951aa7-184d-464a-bfd2-3f606d56e502 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.698358 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1252/2509] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:05 2022] PUT /identity/v3/projects/a1b3b1f26e134ed4a63e15242a08a1a1/users/1755e4d8862d4811a0909ddd6f26dbc0/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:05.709663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74feae-3f69-4a83-a868-873511ccfe6a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.709972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74feae-3f69-4a83-a868-873511ccfe6a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.710292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74feae-3f69-4a83-a868-873511ccfe6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.747491 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9a74feae-3f69-4a83-a868-873511ccfe6a None None] MFA Rules not processed for user `1755e4d8862d4811a0909ddd6f26dbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:05.748771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9a74feae-3f69-4a83-a868-873511ccfe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:05.821789 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1258/2510] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:05.833932 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63951aa7-184d-464a-bfd2-3f606d56e502 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:05.836131 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-63951aa7-184d-464a-bfd2-3f606d56e502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:05.897036 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63951aa7-184d-464a-bfd2-3f606d56e502 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:05.902022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c19a2b-4a94-4391-85e2-4fc66e16d0c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:05.902636 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c19a2b-4a94-4391-85e2-4fc66e16d0c2 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:05.902944 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c19a2b-4a94-4391-85e2-4fc66e16d0c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:05.903217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c19a2b-4a94-4391-85e2-4fc66e16d0c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:05.914104 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c19a2b-4a94-4391-85e2-4fc66e16d0c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:05.915599 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c19a2b-4a94-4391-85e2-4fc66e16d0c2 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:05.926036 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1253/2511] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:06.002813 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74feae-3f69-4a83-a868-873511ccfe6a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:06.013455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74feae-3f69-4a83-a868-873511ccfe6a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:06.018383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355287d9-577d-48fe-9948-35c176e820ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:06.019165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355287d9-577d-48fe-9948-35c176e820ee service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:06.019455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355287d9-577d-48fe-9948-35c176e820ee service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:06.019697 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355287d9-577d-48fe-9948-35c176e820ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:06.069912 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:06.082197 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355287d9-577d-48fe-9948-35c176e820ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:06.083702 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355287d9-577d-48fe-9948-35c176e820ee service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:06.092728 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1259/2512] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:06.367350 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c19a2b-4a94-4391-85e2-4fc66e16d0c2 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:06.430409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c19a2b-4a94-4391-85e2-4fc66e16d0c2 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:06.435108 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:06.435748 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:06.436001 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:06.436323 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] PATH_INFO: `/v3/users/e91cf10a974b4f25977e34213d0bf206` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:06.437534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e91cf10a974b4f25977e34213d0bf206)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:06.439532 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:06.486204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e91cf10a974b4f25977e34213d0bf206'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:06.486909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e91cf10a974b4f25977e34213d0bf206'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:06.494087 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e91cf10a974b4f25977e34213d0bf206'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:06.502184 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e91cf10a974b4f25977e34213d0bf206'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:06.514698 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e91cf10a974b4f25977e34213d0bf206'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:06.520957 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1254/2513] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:57:06 2022] DELETE /identity/v3/users/e91cf10a974b4f25977e34213d0bf206 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:06.526129 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f6345-8058-4e93-9a2f-097df7b0ae7f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:06.526758 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f6345-8058-4e93-9a2f-097df7b0ae7f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:06.527070 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f6345-8058-4e93-9a2f-097df7b0ae7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:06.547088 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-161f6345-8058-4e93-9a2f-097df7b0ae7f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:06.548457 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-161f6345-8058-4e93-9a2f-097df7b0ae7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:06.609828 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1260/2514] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:06.620672 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:06.622209 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:06.629781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:06.687863 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75aa27c-f7b0-4002-bb89-6c5fa8ce00d8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:06.691825 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bbc8e-e918-481e-b4df-68e58f06fe93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:06.692472 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bbc8e-e918-481e-b4df-68e58f06fe93 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:06.692751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bbc8e-e918-481e-b4df-68e58f06fe93 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:06.693022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bbc8e-e918-481e-b4df-68e58f06fe93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:06.703378 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bbc8e-e918-481e-b4df-68e58f06fe93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:06.704822 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bbc8e-e918-481e-b4df-68e58f06fe93 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:06.715196 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1255/2515] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:06.992078 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f6345-8058-4e93-9a2f-097df7b0ae7f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:06.993529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-161f6345-8058-4e93-9a2f-097df7b0ae7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:07.007250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f6345-8058-4e93-9a2f-097df7b0ae7f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:07.012174 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:07.012821 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:07.013096 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:07.013865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] PATH_INFO: `/v3/projects/243b1115c22b4a8ba4d94ccb5aa7d770` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:07.015265 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243b1115c22b4a8ba4d94ccb5aa7d770)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:07.017216 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:07.065771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243b1115c22b4a8ba4d94ccb5aa7d770', 'request_id': 'req-8351590a-d883-4303-ac57-eae3ab319dd1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.071125 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243b1115c22b4a8ba4d94ccb5aa7d770', 'request_id': 'req-8351590a-d883-4303-ac57-eae3ab319dd1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.077065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243b1115c22b4a8ba4d94ccb5aa7d770', 'request_id': 'req-8351590a-d883-4303-ac57-eae3ab319dd1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.089886 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] The token cache is being invalidate because project 243b1115c22b4a8ba4d94ccb5aa7d770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:57:07.090169 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.090794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.092263 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1261/2516] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:57:06 2022] DELETE /identity/v3/projects/243b1115c22b4a8ba4d94ccb5aa7d770 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:07.096320 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bbc8e-e918-481e-b4df-68e58f06fe93 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:07.106096 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bbc8e-e918-481e-b4df-68e58f06fe93 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:07.167009 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:07.167746 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:07.168139 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:07.168479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] PATH_INFO: `/v3/users/1755e4d8862d4811a0909ddd6f26dbc0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:07.169689 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1755e4d8862d4811a0909ddd6f26dbc0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:07.171694 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:07.220396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1755e4d8862d4811a0909ddd6f26dbc0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.221040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1755e4d8862d4811a0909ddd6f26dbc0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.228181 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1755e4d8862d4811a0909ddd6f26dbc0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.236971 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1755e4d8862d4811a0909ddd6f26dbc0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.251079 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f9870526-c8e3-4711-8489-da0bac4ca418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1755e4d8862d4811a0909ddd6f26dbc0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.258037 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1256/2517] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:57:07 2022] DELETE /identity/v3/users/1755e4d8862d4811a0909ddd6f26dbc0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:07.534075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:07.535669 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:07.551222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351590a-d883-4303-ac57-eae3ab319dd1 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:07.630600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:07.631483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:07.631830 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:07.632171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] PATH_INFO: `/v3/projects/a1b3b1f26e134ed4a63e15242a08a1a1` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:07.633542 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b3b1f26e134ed4a63e15242a08a1a1)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:07.636438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:07.653447 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:07.689099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b3b1f26e134ed4a63e15242a08a1a1', 'request_id': 'req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.694964 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b3b1f26e134ed4a63e15242a08a1a1', 'request_id': 'req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.701211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b3b1f26e134ed4a63e15242a08a1a1', 'request_id': 'req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.715494 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] The token cache is being invalidate because project a1b3b1f26e134ed4a63e15242a08a1a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:57:07.715778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.716543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78a14baa-3c9b-4b83-a88c-ed4853a6a8f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:07.718167 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1262/2518] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:57:07 2022] DELETE /identity/v3/projects/a1b3b1f26e134ed4a63e15242a08a1a1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:07.723192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693d54d-0103-4b89-9903-b2312b1ccdc3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:07.723670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693d54d-0103-4b89-9903-b2312b1ccdc3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:07.723788 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693d54d-0103-4b89-9903-b2312b1ccdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:07.746201 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9693d54d-0103-4b89-9903-b2312b1ccdc3 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:07.747460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9693d54d-0103-4b89-9903-b2312b1ccdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:07.770051 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:07.816638 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1257/2519] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 08:57:07.824139 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e370115-1492-4a54-8230-b15cd6e2a642 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:07.824429 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e370115-1492-4a54-8230-b15cd6e2a642 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:07.824691 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e370115-1492-4a54-8230-b15cd6e2a642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:07.845657 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9e370115-1492-4a54-8230-b15cd6e2a642 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:07.847128 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9e370115-1492-4a54-8230-b15cd6e2a642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:07.854813 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1263/2520] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:07.858909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693d54d-0103-4b89-9903-b2312b1ccdc3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:07.868401 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693d54d-0103-4b89-9903-b2312b1ccdc3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:07.872922 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbf5c7-1f70-4379-a514-a19c5db03acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:07.873534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbf5c7-1f70-4379-a514-a19c5db03acd admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:07.873803 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbf5c7-1f70-4379-a514-a19c5db03acd admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:07.874073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbf5c7-1f70-4379-a514-a19c5db03acd admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:07.874741 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbf5c7-1f70-4379-a514-a19c5db03acd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:57:07.875029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbf5c7-1f70-4379-a514-a19c5db03acd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:07.876647 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbf5c7-1f70-4379-a514-a19c5db03acd admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:07.888761 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1258/2521] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:57:07.893064 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e370115-1492-4a54-8230-b15cd6e2a642 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:07.902321 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e370115-1492-4a54-8230-b15cd6e2a642 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:07.907746 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30026531-0ebc-4787-9506-9b82f04c4793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:07.908439 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30026531-0ebc-4787-9506-9b82f04c4793 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:07.908831 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30026531-0ebc-4787-9506-9b82f04c4793 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:07.909157 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30026531-0ebc-4787-9506-9b82f04c4793 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:07.909844 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30026531-0ebc-4787-9506-9b82f04c4793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:07.911820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30026531-0ebc-4787-9506-9b82f04c4793 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:07.943233 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1264/2522] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:57:07 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:07.944445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbf5c7-1f70-4379-a514-a19c5db03acd admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:07.954969 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbf5c7-1f70-4379-a514-a19c5db03acd admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.012496 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb2d49-5063-4c23-b93e-afdb2b34fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.013132 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb2d49-5063-4c23-b93e-afdb2b34fef0 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.013381 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb2d49-5063-4c23-b93e-afdb2b34fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.013682 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb2d49-5063-4c23-b93e-afdb2b34fef0 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.014279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb2d49-5063-4c23-b93e-afdb2b34fef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.016246 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb2d49-5063-4c23-b93e-afdb2b34fef0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.038504 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1259/2523] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:57:07 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:08.042779 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30026531-0ebc-4787-9506-9b82f04c4793 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.053385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30026531-0ebc-4787-9506-9b82f04c4793 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.059040 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605d6e-182b-4786-88a5-92810c829755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.059713 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605d6e-182b-4786-88a5-92810c829755 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.059993 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605d6e-182b-4786-88a5-92810c829755 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.060265 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605d6e-182b-4786-88a5-92810c829755 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.060914 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b605d6e-182b-4786-88a5-92810c829755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.063035 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b605d6e-182b-4786-88a5-92810c829755 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.074492 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1265/2524] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 08:57:08.078685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb2d49-5063-4c23-b93e-afdb2b34fef0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.089215 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb2d49-5063-4c23-b93e-afdb2b34fef0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.094535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351507fd-2c28-4047-b446-56b048aabd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.095188 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351507fd-2c28-4047-b446-56b048aabd85 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.095470 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351507fd-2c28-4047-b446-56b048aabd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.095744 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351507fd-2c28-4047-b446-56b048aabd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.096386 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351507fd-2c28-4047-b446-56b048aabd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.097985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351507fd-2c28-4047-b446-56b048aabd85 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.108780 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1260/2525] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:08.114063 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605d6e-182b-4786-88a5-92810c829755 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.126441 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605d6e-182b-4786-88a5-92810c829755 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.135155 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fde47-ec14-4285-9b82-a4ab6a8dc39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.136141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fde47-ec14-4285-9b82-a4ab6a8dc39f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.136543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fde47-ec14-4285-9b82-a4ab6a8dc39f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.136924 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fde47-ec14-4285-9b82-a4ab6a8dc39f admin admin] PATH_INFO: `/v3/projects/3d014b20776f43e58c92f776c0aadffd/users/d4cf0b74a7db4724a81ac1b3bf249e62/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.156957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fde47-ec14-4285-9b82-a4ab6a8dc39f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d014b20776f43e58c92f776c0aadffd, user_id=d4cf0b74a7db4724a81ac1b3bf249e62, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.161604 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fde47-ec14-4285-9b82-a4ab6a8dc39f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.179800 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1266/2526] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:08 2022] PUT /identity/v3/projects/3d014b20776f43e58c92f776c0aadffd/users/d4cf0b74a7db4724a81ac1b3bf249e62/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:08.191296 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edeb19c-25c8-4a3b-91fd-7ecddecee031 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.191632 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edeb19c-25c8-4a3b-91fd-7ecddecee031 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.191930 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edeb19c-25c8-4a3b-91fd-7ecddecee031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.220817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3edeb19c-25c8-4a3b-91fd-7ecddecee031 None None] MFA Rules not processed for user `d4cf0b74a7db4724a81ac1b3bf249e62`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:08.221934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3edeb19c-25c8-4a3b-91fd-7ecddecee031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:08.290012 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1261/2527] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:08.294661 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fde47-ec14-4285-9b82-a4ab6a8dc39f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.356487 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fde47-ec14-4285-9b82-a4ab6a8dc39f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.361599 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf77f4-4ed6-4c19-be53-1241755576c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.362199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf77f4-4ed6-4c19-be53-1241755576c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.362475 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf77f4-4ed6-4c19-be53-1241755576c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.362751 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf77f4-4ed6-4c19-be53-1241755576c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.363389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf77f4-4ed6-4c19-be53-1241755576c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.365268 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf77f4-4ed6-4c19-be53-1241755576c6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.394465 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1267/2528] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:57:08 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:08.398706 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edeb19c-25c8-4a3b-91fd-7ecddecee031 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.409431 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edeb19c-25c8-4a3b-91fd-7ecddecee031 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.427923 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:08.467008 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d147d0b-0bce-4905-beed-dda7a1facba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.467614 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d147d0b-0bce-4905-beed-dda7a1facba5 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.467889 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d147d0b-0bce-4905-beed-dda7a1facba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.468161 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d147d0b-0bce-4905-beed-dda7a1facba5 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.468781 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d147d0b-0bce-4905-beed-dda7a1facba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.470742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d147d0b-0bce-4905-beed-dda7a1facba5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.493367 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1262/2529] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:57:08 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:08.497202 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf77f4-4ed6-4c19-be53-1241755576c6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.507471 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf77f4-4ed6-4c19-be53-1241755576c6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.512505 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852804c8-e12a-4765-87a8-dfdbbeb2195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.513108 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852804c8-e12a-4765-87a8-dfdbbeb2195d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.513383 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852804c8-e12a-4765-87a8-dfdbbeb2195d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.513778 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852804c8-e12a-4765-87a8-dfdbbeb2195d admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.514432 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852804c8-e12a-4765-87a8-dfdbbeb2195d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.516165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852804c8-e12a-4765-87a8-dfdbbeb2195d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.527315 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1268/2530] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:08.531641 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d147d0b-0bce-4905-beed-dda7a1facba5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.541928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d147d0b-0bce-4905-beed-dda7a1facba5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.547487 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3d9c5-a726-4204-9a1f-28e9ed81e30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.548133 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3d9c5-a726-4204-9a1f-28e9ed81e30a admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.548412 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3d9c5-a726-4204-9a1f-28e9ed81e30a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.548684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3d9c5-a726-4204-9a1f-28e9ed81e30a admin admin] PATH_INFO: `/v3/projects/43eb8489ca374787b3441d7c0311cbdf/users/85ab43247f9c4bc7a43d82fc356461c1/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.564823 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3d9c5-a726-4204-9a1f-28e9ed81e30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=43eb8489ca374787b3441d7c0311cbdf, user_id=85ab43247f9c4bc7a43d82fc356461c1, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.567780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3d9c5-a726-4204-9a1f-28e9ed81e30a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.583873 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1263/2531] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:08 2022] PUT /identity/v3/projects/43eb8489ca374787b3441d7c0311cbdf/users/85ab43247f9c4bc7a43d82fc356461c1/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:08.587974 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852804c8-e12a-4765-87a8-dfdbbeb2195d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.598200 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852804c8-e12a-4765-87a8-dfdbbeb2195d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.607950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:08.659362 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29b8e0-6fba-422a-a580-47fdaaee5597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.660020 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29b8e0-6fba-422a-a580-47fdaaee5597 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.660227 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29b8e0-6fba-422a-a580-47fdaaee5597 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.660498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29b8e0-6fba-422a-a580-47fdaaee5597 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.661134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29b8e0-6fba-422a-a580-47fdaaee5597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.662743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29b8e0-6fba-422a-a580-47fdaaee5597 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.672498 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1269/2532] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:08.677184 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3d9c5-a726-4204-9a1f-28e9ed81e30a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.687897 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3d9c5-a726-4204-9a1f-28e9ed81e30a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.692730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1e706-8950-4725-8ed3-a600b9023112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.693375 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1e706-8950-4725-8ed3-a600b9023112 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.693655 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1e706-8950-4725-8ed3-a600b9023112 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.693928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1e706-8950-4725-8ed3-a600b9023112 admin admin] PATH_INFO: `/v3/projects/43eb8489ca374787b3441d7c0311cbdf/users/85ab43247f9c4bc7a43d82fc356461c1/roles/eadba215d7ee4202b9fb56ea79c33b11` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.695914 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1e706-8950-4725-8ed3-a600b9023112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43eb8489ca374787b3441d7c0311cbdf, user_id=85ab43247f9c4bc7a43d82fc356461c1, role_id=eadba215d7ee4202b9fb56ea79c33b11)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.698528 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1e706-8950-4725-8ed3-a600b9023112 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.709287 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1264/2533] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:08 2022] PUT /identity/v3/projects/43eb8489ca374787b3441d7c0311cbdf/users/85ab43247f9c4bc7a43d82fc356461c1/roles/eadba215d7ee4202b9fb56ea79c33b11 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:08.719536 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9f988-992a-4feb-b4fa-af654496c48c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.719820 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9f988-992a-4feb-b4fa-af654496c48c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.720085 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9f988-992a-4feb-b4fa-af654496c48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.749990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bef9f988-992a-4feb-b4fa-af654496c48c None None] MFA Rules not processed for user `85ab43247f9c4bc7a43d82fc356461c1`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:08.751164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bef9f988-992a-4feb-b4fa-af654496c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:08.821648 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1270/2534] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:08.832603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1e706-8950-4725-8ed3-a600b9023112 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:08.834351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c4a1e706-8950-4725-8ed3-a600b9023112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:08.893357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1e706-8950-4725-8ed3-a600b9023112 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:08.897334 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbf19a-56f0-4f5f-aec1-e90cf6d4a1ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:08.897926 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbf19a-56f0-4f5f-aec1-e90cf6d4a1ea service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:08.898204 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbf19a-56f0-4f5f-aec1-e90cf6d4a1ea service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:08.898475 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbf19a-56f0-4f5f-aec1-e90cf6d4a1ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:08.957284 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbf19a-56f0-4f5f-aec1-e90cf6d4a1ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:08.958708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbf19a-56f0-4f5f-aec1-e90cf6d4a1ea service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:08.968997 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1265/2535] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:13.221829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9f988-992a-4feb-b4fa-af654496c48c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:13.223449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bef9f988-992a-4feb-b4fa-af654496c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:13.236606 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9f988-992a-4feb-b4fa-af654496c48c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:13.240224 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589c523-ccae-4429-92c9-4ee4eec3e750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:13.240832 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589c523-ccae-4429-92c9-4ee4eec3e750 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:13.241109 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589c523-ccae-4429-92c9-4ee4eec3e750 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:13.241385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589c523-ccae-4429-92c9-4ee4eec3e750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:13.251586 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589c523-ccae-4429-92c9-4ee4eec3e750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:13.253013 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589c523-ccae-4429-92c9-4ee4eec3e750 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:13.263256 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1271/2536] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:14.184585 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513b52b-9122-4d23-8387-aeef476b2882 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:14.185173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513b52b-9122-4d23-8387-aeef476b2882 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:14.185173 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513b52b-9122-4d23-8387-aeef476b2882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:14.204928 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7513b52b-9122-4d23-8387-aeef476b2882 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:14.206112 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7513b52b-9122-4d23-8387-aeef476b2882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:14.231982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:14.268814 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1266/2537] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:57:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:14.313652 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589c523-ccae-4429-92c9-4ee4eec3e750 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:14.315237 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7589c523-ccae-4429-92c9-4ee4eec3e750 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:14.382379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589c523-ccae-4429-92c9-4ee4eec3e750 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:14.387593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e00fe-b06e-499d-a32a-4bc535cfb41f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:14.388194 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e00fe-b06e-499d-a32a-4bc535cfb41f service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:14.388468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e00fe-b06e-499d-a32a-4bc535cfb41f service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:14.388737 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e00fe-b06e-499d-a32a-4bc535cfb41f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:14.396987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e00fe-b06e-499d-a32a-4bc535cfb41f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:14.398545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e00fe-b06e-499d-a32a-4bc535cfb41f service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:14.408512 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1272/2538] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:57:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:57:14.923124 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0154252-a719-43b0-ab81-0b653f567c8c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:14.923453 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0154252-a719-43b0-ab81-0b653f567c8c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:14.923751 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0154252-a719-43b0-ab81-0b653f567c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:14.944727 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e0154252-a719-43b0-ab81-0b653f567c8c None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:14.945935 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e0154252-a719-43b0-ab81-0b653f567c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:14.954517 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1267/2539] 10.222.0.26 () {58 vars in 967 bytes} [Mon Oct 24 08:57:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:14.966942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e00fe-b06e-499d-a32a-4bc535cfb41f service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:14.977884 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e00fe-b06e-499d-a32a-4bc535cfb41f service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:14.983947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2f7f7-b019-43ab-80b5-dfb22834e1e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:14.984614 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f7f7-b019-43ab-80b5-dfb22834e1e6 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:14.984899 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f7f7-b019-43ab-80b5-dfb22834e1e6 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:14.985171 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f7f7-b019-43ab-80b5-dfb22834e1e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:14.994345 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2f7f7-b019-43ab-80b5-dfb22834e1e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:14.996143 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2f7f7-b019-43ab-80b5-dfb22834e1e6 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:14.999736 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:15.005955 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1273/2540] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:57:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:57:15.258530 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450105f-92c3-40c6-964d-291135c479f9 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:15.259094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450105f-92c3-40c6-964d-291135c479f9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:15.259094 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450105f-92c3-40c6-964d-291135c479f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:15.280088 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a450105f-92c3-40c6-964d-291135c479f9 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:15.281328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a450105f-92c3-40c6-964d-291135c479f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:15.289740 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1268/2541] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Oct 24 08:57:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:15.304052 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2f7f7-b019-43ab-80b5-dfb22834e1e6 service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:15.315828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2f7f7-b019-43ab-80b5-dfb22834e1e6 service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:15.323047 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3b689-1370-4645-86ee-37f58317d85e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:15.323658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3b689-1370-4645-86ee-37f58317d85e service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:15.323958 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3b689-1370-4645-86ee-37f58317d85e service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:15.324222 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3b689-1370-4645-86ee-37f58317d85e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:15.333065 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3b689-1370-4645-86ee-37f58317d85e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:15.334741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3b689-1370-4645-86ee-37f58317d85e service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:15.343179 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1274/2542] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:57:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:57:15.677700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd908-cd86-45fa-abb6-5a444d8a5af3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:15.677983 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd908-cd86-45fa-abb6-5a444d8a5af3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:15.678258 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd908-cd86-45fa-abb6-5a444d8a5af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:15.697817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7ddfd908-cd86-45fa-abb6-5a444d8a5af3 None None] MFA Rules not processed for user `771fc0da7a8749f0b80ce0630951500a`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:15.698963 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7ddfd908-cd86-45fa-abb6-5a444d8a5af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:15.706617 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1269/2543] 10.222.0.26 () {58 vars in 967 bytes} [Mon Oct 24 08:57:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:15.719191 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3b689-1370-4645-86ee-37f58317d85e service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:15.730303 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3b689-1370-4645-86ee-37f58317d85e service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:15.736289 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de099d-7d37-4b90-b3e1-921f1fa45629 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf288ceecac4fec8b2161c55a61a8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:15.736882 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de099d-7d37-4b90-b3e1-921f1fa45629 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:15.737154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de099d-7d37-4b90-b3e1-921f1fa45629 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:15.737425 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de099d-7d37-4b90-b3e1-921f1fa45629 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:15.745738 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de099d-7d37-4b90-b3e1-921f1fa45629 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:15.747338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de099d-7d37-4b90-b3e1-921f1fa45629 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:15.755482 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1275/2544] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Oct 24 08:57:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 08:57:17.913728 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfd908-cd86-45fa-abb6-5a444d8a5af3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:17.915385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7ddfd908-cd86-45fa-abb6-5a444d8a5af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:17.988092 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfd908-cd86-45fa-abb6-5a444d8a5af3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:17.993490 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:17.994106 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:17.994380 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:17.994629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] PATH_INFO: `/v3/users/8926edfa5bbd48ae91bca6e01dd0e642` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:17.995794 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8926edfa5bbd48ae91bca6e01dd0e642)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:17.997701 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:18.048037 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8926edfa5bbd48ae91bca6e01dd0e642'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.048676 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8926edfa5bbd48ae91bca6e01dd0e642'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.056190 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8926edfa5bbd48ae91bca6e01dd0e642'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.064709 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8926edfa5bbd48ae91bca6e01dd0e642'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.076264 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8926edfa5bbd48ae91bca6e01dd0e642'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.081933 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1270/2545] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:57:17 2022] DELETE /identity/v3/users/8926edfa5bbd48ae91bca6e01dd0e642 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:18.086841 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281b01f-6af1-4c8b-b737-f452d4f49f1a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:18.087338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281b01f-6af1-4c8b-b737-f452d4f49f1a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:18.087434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281b01f-6af1-4c8b-b737-f452d4f49f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:18.107818 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c281b01f-6af1-4c8b-b737-f452d4f49f1a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:18.108989 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c281b01f-6af1-4c8b-b737-f452d4f49f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:18.173535 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:18.175774 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:18.178037 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1276/2546] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:18.187578 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281b01f-6af1-4c8b-b737-f452d4f49f1a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:18.188987 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c281b01f-6af1-4c8b-b737-f452d4f49f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:18.189292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a5283-5638-4155-890b-88082cb4af99 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:18.194211 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:18.194894 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:18.195177 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:18.195442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] PATH_INFO: `/v3/users/d4cf0b74a7db4724a81ac1b3bf249e62` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:18.196629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4cf0b74a7db4724a81ac1b3bf249e62)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:18.198670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:18.230536 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:18.250016 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4cf0b74a7db4724a81ac1b3bf249e62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.250667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4cf0b74a7db4724a81ac1b3bf249e62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.253530 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281b01f-6af1-4c8b-b737-f452d4f49f1a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:18.257117 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118377f-7f5f-4a03-b556-48caab748a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:18.257757 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4cf0b74a7db4724a81ac1b3bf249e62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.257868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118377f-7f5f-4a03-b556-48caab748a5b service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:18.258033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118377f-7f5f-4a03-b556-48caab748a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:18.258301 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118377f-7f5f-4a03-b556-48caab748a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:18.265098 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4cf0b74a7db4724a81ac1b3bf249e62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.273725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118377f-7f5f-4a03-b556-48caab748a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:18.275249 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118377f-7f5f-4a03-b556-48caab748a5b service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:18.280495 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4cf0b74a7db4724a81ac1b3bf249e62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.285805 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1277/2547] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 08:57:18.287520 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1271/2548] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:57:18 2022] DELETE /identity/v3/users/d4cf0b74a7db4724a81ac1b3bf249e62 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:57:18.291601 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f3b38-3f14-45f3-8ac8-b0404c2054d8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:18.291885 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f3b38-3f14-45f3-8ac8-b0404c2054d8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:18.292156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f3b38-3f14-45f3-8ac8-b0404c2054d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:18.312305 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-db0f3b38-3f14-45f3-8ac8-b0404c2054d8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:18.313498 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-db0f3b38-3f14-45f3-8ac8-b0404c2054d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:18.371487 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1278/2549] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:18.381500 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:18.382925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:18.441999 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f8776-07cd-4015-b32e-ff02664247da admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:18.447673 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ff709-6691-44aa-8c82-48f148fb457e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:18.448455 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ff709-6691-44aa-8c82-48f148fb457e service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:18.448732 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ff709-6691-44aa-8c82-48f148fb457e service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:18.449023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ff709-6691-44aa-8c82-48f148fb457e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:18.459251 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ff709-6691-44aa-8c82-48f148fb457e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:18.459726 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f3b38-3f14-45f3-8ac8-b0404c2054d8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:18.460737 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ff709-6691-44aa-8c82-48f148fb457e service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:18.461254 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-db0f3b38-3f14-45f3-8ac8-b0404c2054d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:18.470784 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1272/2550] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:18.486472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f3b38-3f14-45f3-8ac8-b0404c2054d8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:18.491695 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:18.492590 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:18.492990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:18.493357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] PATH_INFO: `/v3/projects/77df0bbca3c740b789fe7e0ea62f9e3a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:18.494635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] RBAC: Authorizing `identity:delete_project(project_id=77df0bbca3c740b789fe7e0ea62f9e3a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:18.496678 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:18.511462 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:18.552510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77df0bbca3c740b789fe7e0ea62f9e3a', 'request_id': 'req-7fd75fc8-e10b-4722-9009-900bb8e851db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.557943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77df0bbca3c740b789fe7e0ea62f9e3a', 'request_id': 'req-7fd75fc8-e10b-4722-9009-900bb8e851db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.564245 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77df0bbca3c740b789fe7e0ea62f9e3a', 'request_id': 'req-7fd75fc8-e10b-4722-9009-900bb8e851db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.577585 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] The token cache is being invalidate because project 77df0bbca3c740b789fe7e0ea62f9e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:57:18.577896 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.578520 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.580109 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1279/2551] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:57:18 2022] DELETE /identity/v3/projects/77df0bbca3c740b789fe7e0ea62f9e3a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:57:18.771288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ff709-6691-44aa-8c82-48f148fb457e service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:18.772811 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-996ff709-6691-44aa-8c82-48f148fb457e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:18.788958 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ff709-6691-44aa-8c82-48f148fb457e service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:18.850994 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:18.851620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:18.851895 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:18.852163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] PATH_INFO: `/v3/projects/3d014b20776f43e58c92f776c0aadffd` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:18.853289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d014b20776f43e58c92f776c0aadffd)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:18.855208 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:18.910396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d014b20776f43e58c92f776c0aadffd', 'request_id': 'req-7f181bac-d489-41d1-b9d7-b4c33baae21c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.911572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:18.917881 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d014b20776f43e58c92f776c0aadffd', 'request_id': 'req-7f181bac-d489-41d1-b9d7-b4c33baae21c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.925439 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d014b20776f43e58c92f776c0aadffd', 'request_id': 'req-7f181bac-d489-41d1-b9d7-b4c33baae21c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.939392 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] The token cache is being invalidate because project 3d014b20776f43e58c92f776c0aadffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:57:18.939671 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.940288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:18.941746 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1273/2552] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:57:18 2022] DELETE /identity/v3/projects/3d014b20776f43e58c92f776c0aadffd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:18.946241 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:18.947684 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:18.960581 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd75fc8-e10b-4722-9009-900bb8e851db admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:19.020027 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:19.020658 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.020942 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.021211 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] PATH_INFO: `/v3/users/85ab43247f9c4bc7a43d82fc356461c1` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:19.022376 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ab43247f9c4bc7a43d82fc356461c1)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:19.024301 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:19.073338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ab43247f9c4bc7a43d82fc356461c1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.080250 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ab43247f9c4bc7a43d82fc356461c1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.080895 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ab43247f9c4bc7a43d82fc356461c1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.089368 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ab43247f9c4bc7a43d82fc356461c1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.102320 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5ffd88ad-1c7d-4370-8e09-ca8bf99823e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ab43247f9c4bc7a43d82fc356461c1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.108319 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1280/2553] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:57:18 2022] DELETE /identity/v3/users/85ab43247f9c4bc7a43d82fc356461c1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:19.403847 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:19.405273 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:19.421351 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f181bac-d489-41d1-b9d7-b4c33baae21c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:19.481085 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:19.481635 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.481909 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.482179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] PATH_INFO: `/v3/projects/43eb8489ca374787b3441d7c0311cbdf` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:19.483302 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43eb8489ca374787b3441d7c0311cbdf)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:19.485216 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:19.539503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43eb8489ca374787b3441d7c0311cbdf', 'request_id': 'req-f08195ae-59ed-4de9-b096-4c2bc086e529'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.545814 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43eb8489ca374787b3441d7c0311cbdf', 'request_id': 'req-f08195ae-59ed-4de9-b096-4c2bc086e529'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.552547 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43eb8489ca374787b3441d7c0311cbdf', 'request_id': 'req-f08195ae-59ed-4de9-b096-4c2bc086e529'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.566265 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] The token cache is being invalidate because project 43eb8489ca374787b3441d7c0311cbdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:57:19.566554 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.567210 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f08195ae-59ed-4de9-b096-4c2bc086e529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:57:19.568688 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1274/2554] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:57:19 2022] DELETE /identity/v3/projects/43eb8489ca374787b3441d7c0311cbdf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:57:19.574709 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12713db3-e66e-402a-9ae6-73eeabf57242 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.575039 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12713db3-e66e-402a-9ae6-73eeabf57242 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.575338 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12713db3-e66e-402a-9ae6-73eeabf57242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:19.595967 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-12713db3-e66e-402a-9ae6-73eeabf57242 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:19.597149 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-12713db3-e66e-402a-9ae6-73eeabf57242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:19.610274 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:19.659450 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1281/2555] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:19.669118 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b409b42-2e60-4d2c-a559-c403d166a5df None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.669405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b409b42-2e60-4d2c-a559-c403d166a5df None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.669681 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b409b42-2e60-4d2c-a559-c403d166a5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:19.691183 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0b409b42-2e60-4d2c-a559-c403d166a5df None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:19.692377 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0b409b42-2e60-4d2c-a559-c403d166a5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:19.700535 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1275/2556] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:19.704776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12713db3-e66e-402a-9ae6-73eeabf57242 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:19.714688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12713db3-e66e-402a-9ae6-73eeabf57242 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:19.720473 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7330a-65e0-44f2-b2c7-dd574e8d5019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:19.721080 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7330a-65e0-44f2-b2c7-dd574e8d5019 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.721360 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7330a-65e0-44f2-b2c7-dd574e8d5019 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.721635 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7330a-65e0-44f2-b2c7-dd574e8d5019 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:19.722275 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7330a-65e0-44f2-b2c7-dd574e8d5019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:57:19.722589 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7330a-65e0-44f2-b2c7-dd574e8d5019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:19.724214 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b7330a-65e0-44f2-b2c7-dd574e8d5019 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:19.736338 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1282/2557] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 08:57:19.740691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b409b42-2e60-4d2c-a559-c403d166a5df None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:19.750251 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b409b42-2e60-4d2c-a559-c403d166a5df None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:19.756314 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2e915-dfa3-4c38-8696-7067a5764a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:19.756902 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2e915-dfa3-4c38-8696-7067a5764a9e admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.757179 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2e915-dfa3-4c38-8696-7067a5764a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.757445 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2e915-dfa3-4c38-8696-7067a5764a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:19.758073 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2e915-dfa3-4c38-8696-7067a5764a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:19.759982 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2e915-dfa3-4c38-8696-7067a5764a9e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:19.789784 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1276/2558] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:57:19 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:57:19.795299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7330a-65e0-44f2-b2c7-dd574e8d5019 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:19.805642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7330a-65e0-44f2-b2c7-dd574e8d5019 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:19.866802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd7879-ad1e-40d7-a972-358a1071a0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:19.867667 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd7879-ad1e-40d7-a972-358a1071a0ad admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.868033 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd7879-ad1e-40d7-a972-358a1071a0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.868409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd7879-ad1e-40d7-a972-358a1071a0ad admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:19.869534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd7879-ad1e-40d7-a972-358a1071a0ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:19.872545 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd7879-ad1e-40d7-a972-358a1071a0ad admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:19.901254 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1283/2559] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:57:19 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:57:19.905629 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2e915-dfa3-4c38-8696-7067a5764a9e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:19.915870 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2e915-dfa3-4c38-8696-7067a5764a9e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:19.920620 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed80699-5804-49a8-8781-98ae4eb1b015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:19.921254 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed80699-5804-49a8-8781-98ae4eb1b015 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.921503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed80699-5804-49a8-8781-98ae4eb1b015 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.921773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed80699-5804-49a8-8781-98ae4eb1b015 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:19.922413 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed80699-5804-49a8-8781-98ae4eb1b015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:19.924134 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed80699-5804-49a8-8781-98ae4eb1b015 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:19.934947 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1277/2560] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:19.939080 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd7879-ad1e-40d7-a972-358a1071a0ad admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:19.949616 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd7879-ad1e-40d7-a972-358a1071a0ad admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:19.955490 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeba83-3b9a-432c-bcb2-b6daf614e4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:19.956084 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeba83-3b9a-432c-bcb2-b6daf614e4eb admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.956359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeba83-3b9a-432c-bcb2-b6daf614e4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.956622 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeba83-3b9a-432c-bcb2-b6daf614e4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:19.957255 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeba83-3b9a-432c-bcb2-b6daf614e4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:19.958889 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeba83-3b9a-432c-bcb2-b6daf614e4eb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:19.970007 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1284/2561] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:57:19.974159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed80699-5804-49a8-8781-98ae4eb1b015 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:19.984023 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed80699-5804-49a8-8781-98ae4eb1b015 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:19.988937 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e23a1a-925f-4905-8dd6-a8988d578c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:19.989503 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e23a1a-925f-4905-8dd6-a8988d578c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:19.989775 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e23a1a-925f-4905-8dd6-a8988d578c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:19.990048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e23a1a-925f-4905-8dd6-a8988d578c44 admin admin] PATH_INFO: `/v3/projects/0b1b162d022d4c5ebd8c62dcad277727/users/a49947bfa2a5482d919ba0d4744553c7/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:20.004830 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e23a1a-925f-4905-8dd6-a8988d578c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1b162d022d4c5ebd8c62dcad277727, user_id=a49947bfa2a5482d919ba0d4744553c7, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:20.007466 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e23a1a-925f-4905-8dd6-a8988d578c44 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:20.022485 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1278/2562] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:57:19 2022] PUT /identity/v3/projects/0b1b162d022d4c5ebd8c62dcad277727/users/a49947bfa2a5482d919ba0d4744553c7/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 08:57:20.030213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69406832-e041-4b66-bcf4-341dcd269ca6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:20.030604 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69406832-e041-4b66-bcf4-341dcd269ca6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:20.031127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69406832-e041-4b66-bcf4-341dcd269ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:20.054857 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-69406832-e041-4b66-bcf4-341dcd269ca6 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:20.056048 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-69406832-e041-4b66-bcf4-341dcd269ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:20.117802 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1285/2563] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:20.127773 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e23a1a-925f-4905-8dd6-a8988d578c44 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:20.130055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d8e23a1a-925f-4905-8dd6-a8988d578c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:20.137449 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:20.192278 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e23a1a-925f-4905-8dd6-a8988d578c44 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:20.196742 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54865-9e9a-45ba-8c89-7045195e60f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:20.197340 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54865-9e9a-45ba-8c89-7045195e60f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:20.197622 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54865-9e9a-45ba-8c89-7045195e60f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:20.197867 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd54865-9e9a-45ba-8c89-7045195e60f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:20.208334 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd54865-9e9a-45ba-8c89-7045195e60f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:20.209782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd54865-9e9a-45ba-8c89-7045195e60f7 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:20.222241 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1279/2564] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:20.843393 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf1676-1259-4358-bebe-42f63bc86e14 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:20.843673 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf1676-1259-4358-bebe-42f63bc86e14 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:20.843959 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf1676-1259-4358-bebe-42f63bc86e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:20.871529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e3cf1676-1259-4358-bebe-42f63bc86e14 None None] MFA Rules not processed for user `a49947bfa2a5482d919ba0d4744553c7`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:20.872743 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e3cf1676-1259-4358-bebe-42f63bc86e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:20.884977 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:20.941457 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1286/2565] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:20.953040 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54865-9e9a-45ba-8c89-7045195e60f7 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:20.954523 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-edd54865-9e9a-45ba-8c89-7045195e60f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:21.010878 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd54865-9e9a-45ba-8c89-7045195e60f7 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:21.014508 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c431fd4-92eb-4515-9c9b-03693a4b615f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:21.015140 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c431fd4-92eb-4515-9c9b-03693a4b615f service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:21.015418 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c431fd4-92eb-4515-9c9b-03693a4b615f service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:21.015685 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c431fd4-92eb-4515-9c9b-03693a4b615f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:21.024045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c431fd4-92eb-4515-9c9b-03693a4b615f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:21.025463 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c431fd4-92eb-4515-9c9b-03693a4b615f service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:21.034241 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1280/2566] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:22.414809 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2c422-f76d-42d9-8a92-3c92c03a7960 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:22.415346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2c422-f76d-42d9-8a92-3c92c03a7960 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:22.415346 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2c422-f76d-42d9-8a92-3c92c03a7960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:22.442091 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a7f2c422-f76d-42d9-8a92-3c92c03a7960 None None] MFA Rules not processed for user `a49947bfa2a5482d919ba0d4744553c7`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:22.443312 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a7f2c422-f76d-42d9-8a92-3c92c03a7960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:22.451503 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1287/2567] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:57:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:31.030504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=50 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:32.435165 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=24 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:57:53.440081 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c431fd4-92eb-4515-9c9b-03693a4b615f service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:53.449137 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c431fd4-92eb-4515-9c9b-03693a4b615f service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:53.453171 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:53.453782 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:53.454058 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:53.454329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:53.456354 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:53.529820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:53.531291 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:53.542367 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1281/2568] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:57:56.559978 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744d926-0f34-494b-a1ba-e7986f0b5b18 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:56.560947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744d926-0f34-494b-a1ba-e7986f0b5b18 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:56.560947 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744d926-0f34-494b-a1ba-e7986f0b5b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:56.593583 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1744d926-0f34-494b-a1ba-e7986f0b5b18 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:57:56.594819 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1744d926-0f34-494b-a1ba-e7986f0b5b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:56.604075 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1288/2569] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:57:56.628135 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:57:56.630292 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:57:56.644880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb88448-4834-4e45-961f-695001957db0 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:57:56.651861 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b55812-0edd-4027-87bb-a8e1148cd765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:57:56.652446 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b55812-0edd-4027-87bb-a8e1148cd765 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:57:56.652721 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b55812-0edd-4027-87bb-a8e1148cd765 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:57:56.652985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b55812-0edd-4027-87bb-a8e1148cd765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:57:56.661583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b55812-0edd-4027-87bb-a8e1148cd765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:57:56.663224 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b55812-0edd-4027-87bb-a8e1148cd765 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:57:56.674573 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1282/2570] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:00.727199 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577ff77-fa5c-4a4f-bac5-0e9d02c8ca66 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:00.727782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577ff77-fa5c-4a4f-bac5-0e9d02c8ca66 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:00.727782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577ff77-fa5c-4a4f-bac5-0e9d02c8ca66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:00.750152 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d577ff77-fa5c-4a4f-bac5-0e9d02c8ca66 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:00.751390 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d577ff77-fa5c-4a4f-bac5-0e9d02c8ca66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:00.759843 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1289/2571] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:00.782854 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b55812-0edd-4027-87bb-a8e1148cd765 service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:00.784289 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-66b55812-0edd-4027-87bb-a8e1148cd765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:00.799521 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b55812-0edd-4027-87bb-a8e1148cd765 service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:00.807434 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bfb2e-a7ea-4e73-bb4a-cb85293aa620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:00.808018 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bfb2e-a7ea-4e73-bb4a-cb85293aa620 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:00.808304 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bfb2e-a7ea-4e73-bb4a-cb85293aa620 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:00.808571 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bfb2e-a7ea-4e73-bb4a-cb85293aa620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:00.817055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bfb2e-a7ea-4e73-bb4a-cb85293aa620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:00.818638 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bfb2e-a7ea-4e73-bb4a-cb85293aa620 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:00.828390 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1283/2572] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:04.028593 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a9ee4-3864-4e15-a463-80715e915082 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:04.029186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a9ee4-3864-4e15-a463-80715e915082 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:04.029186 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a9ee4-3864-4e15-a463-80715e915082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:04.050093 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bc8a9ee4-3864-4e15-a463-80715e915082 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:04.051711 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bc8a9ee4-3864-4e15-a463-80715e915082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:04.061283 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1290/2573] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:04.083467 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bfb2e-a7ea-4e73-bb4a-cb85293aa620 service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:04.095422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bfb2e-a7ea-4e73-bb4a-cb85293aa620 service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:04.102576 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683bf2e-b032-4f2f-b1b1-34ce8a4b285e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:04.103212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683bf2e-b032-4f2f-b1b1-34ce8a4b285e service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:04.103483 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683bf2e-b032-4f2f-b1b1-34ce8a4b285e service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:04.103756 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683bf2e-b032-4f2f-b1b1-34ce8a4b285e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:04.112797 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683bf2e-b032-4f2f-b1b1-34ce8a4b285e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:04.114435 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683bf2e-b032-4f2f-b1b1-34ce8a4b285e service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:04.124573 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1284/2574] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:07.118364 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323332-65da-47d8-89b6-73460889e26f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:07.118948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323332-65da-47d8-89b6-73460889e26f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:07.118948 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323332-65da-47d8-89b6-73460889e26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:07.139537 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-45323332-65da-47d8-89b6-73460889e26f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:07.140748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-45323332-65da-47d8-89b6-73460889e26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:07.149221 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1291/2575] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:07.173376 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683bf2e-b032-4f2f-b1b1-34ce8a4b285e service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:07.187234 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683bf2e-b032-4f2f-b1b1-34ce8a4b285e service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:07.193896 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d9d0f-2088-4a6c-9886-9fe30de95b27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:07.194480 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d9d0f-2088-4a6c-9886-9fe30de95b27 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:07.194777 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d9d0f-2088-4a6c-9886-9fe30de95b27 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:07.195047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d9d0f-2088-4a6c-9886-9fe30de95b27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:07.203561 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d9d0f-2088-4a6c-9886-9fe30de95b27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:07.205121 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d9d0f-2088-4a6c-9886-9fe30de95b27 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:07.214635 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1285/2576] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:11.344815 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dcc5d-3378-4cf1-9117-5432ac4cfaf7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:11.345618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dcc5d-3378-4cf1-9117-5432ac4cfaf7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:11.345618 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dcc5d-3378-4cf1-9117-5432ac4cfaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:11.375800 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-916dcc5d-3378-4cf1-9117-5432ac4cfaf7 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:11.377024 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-916dcc5d-3378-4cf1-9117-5432ac4cfaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:11.386737 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1292/2577] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:11.409156 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d9d0f-2088-4a6c-9886-9fe30de95b27 service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:11.420938 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d9d0f-2088-4a6c-9886-9fe30de95b27 service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:11.428405 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540b50-89a7-4b79-8c9b-fbb12b08c38b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:11.429002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540b50-89a7-4b79-8c9b-fbb12b08c38b service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:11.429279 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540b50-89a7-4b79-8c9b-fbb12b08c38b service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:11.429545 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540b50-89a7-4b79-8c9b-fbb12b08c38b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:11.438420 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d540b50-89a7-4b79-8c9b-fbb12b08c38b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:11.440014 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d540b50-89a7-4b79-8c9b-fbb12b08c38b service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:11.449885 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1286/2578] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:14.521748 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbb55f-c281-4a36-a943-47bc604ebe4f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:14.522294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbb55f-c281-4a36-a943-47bc604ebe4f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:14.522294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbb55f-c281-4a36-a943-47bc604ebe4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:14.544073 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a1cbb55f-c281-4a36-a943-47bc604ebe4f None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:14.545193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a1cbb55f-c281-4a36-a943-47bc604ebe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:14.554648 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1293/2579] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:14.576778 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540b50-89a7-4b79-8c9b-fbb12b08c38b service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:14.589329 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540b50-89a7-4b79-8c9b-fbb12b08c38b service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:14.595892 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5979-b566-4260-a31e-f5e51cf597e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:14.596473 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5979-b566-4260-a31e-f5e51cf597e7 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:14.596772 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5979-b566-4260-a31e-f5e51cf597e7 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:14.597029 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5979-b566-4260-a31e-f5e51cf597e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:14.606626 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad5979-b566-4260-a31e-f5e51cf597e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:14.608927 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad5979-b566-4260-a31e-f5e51cf597e7 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:14.626086 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1287/2580] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:17.659294 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbb55f-c281-4a36-a943-47bc604ebe4f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:17.660854 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a1cbb55f-c281-4a36-a943-47bc604ebe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:17.675611 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbb55f-c281-4a36-a943-47bc604ebe4f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:17.679872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae7a63-fecf-442f-a27e-6ab355ebac19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:17.680468 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae7a63-fecf-442f-a27e-6ab355ebac19 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:17.680741 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae7a63-fecf-442f-a27e-6ab355ebac19 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:17.681009 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae7a63-fecf-442f-a27e-6ab355ebac19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:17.682828 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0eae7a63-fecf-442f-a27e-6ab355ebac19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:17.696309 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae7a63-fecf-442f-a27e-6ab355ebac19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:17.697727 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae7a63-fecf-442f-a27e-6ab355ebac19 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:17.708424 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1294/2581] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:24.459953 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5979-b566-4260-a31e-f5e51cf597e7 service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:24.466242 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=71 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:24.472628 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5979-b566-4260-a31e-f5e51cf597e7 service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:24.478197 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:24.479072 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:24.479546 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:24.479820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:24.481556 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:24.495159 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:24.496776 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:24.507652 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1288/2582] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:27.703990 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=24 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:29.906955 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80624803-2a45-483d-ba4e-dfeecbc75de8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:29.907608 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80624803-2a45-483d-ba4e-dfeecbc75de8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:29.907608 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80624803-2a45-483d-ba4e-dfeecbc75de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:29.928299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-80624803-2a45-483d-ba4e-dfeecbc75de8 None None] MFA Rules not processed for user `a49947bfa2a5482d919ba0d4744553c7`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:29.929496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-80624803-2a45-483d-ba4e-dfeecbc75de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:29.937499 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1295/2583] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:58:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:30.506425 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:30.508077 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:30.580875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc96dd-d190-4308-b54b-79fec312103e service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:30.586422 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1119b-4438-48fe-a892-636cd854d2a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:30.587055 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1119b-4438-48fe-a892-636cd854d2a5 service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:30.587334 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1119b-4438-48fe-a892-636cd854d2a5 service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:30.587607 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1119b-4438-48fe-a892-636cd854d2a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:30.589382 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c1b1119b-4438-48fe-a892-636cd854d2a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:30.600637 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1119b-4438-48fe-a892-636cd854d2a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:30.602229 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1119b-4438-48fe-a892-636cd854d2a5 service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:30.611966 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1289/2584] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:32.425837 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80624803-2a45-483d-ba4e-dfeecbc75de8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:32.427718 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-80624803-2a45-483d-ba4e-dfeecbc75de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:32.440794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80624803-2a45-483d-ba4e-dfeecbc75de8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:32.444921 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f46845-e612-4529-9670-c0a0e0897d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:32.445722 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f46845-e612-4529-9670-c0a0e0897d3a service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:32.446164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f46845-e612-4529-9670-c0a0e0897d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:32.446449 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f46845-e612-4529-9670-c0a0e0897d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:32.448292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-32f46845-e612-4529-9670-c0a0e0897d3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:32.458677 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f46845-e612-4529-9670-c0a0e0897d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:32.460134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f46845-e612-4529-9670-c0a0e0897d3a service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:32.468686 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1296/2585] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:40.607763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=37 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:42.465003 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=9 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:58:43.013311 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759209b5-5029-49ff-9632-0e5355165ad1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:43.013879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759209b5-5029-49ff-9632-0e5355165ad1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:43.013879 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759209b5-5029-49ff-9632-0e5355165ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:43.034345 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-759209b5-5029-49ff-9632-0e5355165ad1 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:43.035566 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-759209b5-5029-49ff-9632-0e5355165ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:43.043934 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1290/2586] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:43.064041 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f46845-e612-4529-9670-c0a0e0897d3a service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:43.077435 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f46845-e612-4529-9670-c0a0e0897d3a service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:43.083170 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266fda-4341-4ea8-848b-b8f532756ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:43.083764 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266fda-4341-4ea8-848b-b8f532756ccc service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:43.084036 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266fda-4341-4ea8-848b-b8f532756ccc service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:43.084297 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266fda-4341-4ea8-848b-b8f532756ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:43.092524 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266fda-4341-4ea8-848b-b8f532756ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:43.094075 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266fda-4341-4ea8-848b-b8f532756ccc service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:43.103379 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1297/2587] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:46.289934 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dce44-7f35-440c-ae8f-cc9b38ac7d5a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:46.290427 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dce44-7f35-440c-ae8f-cc9b38ac7d5a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:46.290552 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dce44-7f35-440c-ae8f-cc9b38ac7d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:46.311537 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ef6dce44-7f35-440c-ae8f-cc9b38ac7d5a None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:46.312725 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ef6dce44-7f35-440c-ae8f-cc9b38ac7d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:46.322644 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1291/2588] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:46.342654 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266fda-4341-4ea8-848b-b8f532756ccc service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:46.354552 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266fda-4341-4ea8-848b-b8f532756ccc service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:46.361637 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28870a34-7efc-424a-a108-431b54b6e908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:46.362236 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28870a34-7efc-424a-a108-431b54b6e908 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:46.362521 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28870a34-7efc-424a-a108-431b54b6e908 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:46.362823 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28870a34-7efc-424a-a108-431b54b6e908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:46.371872 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28870a34-7efc-424a-a108-431b54b6e908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:46.373548 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28870a34-7efc-424a-a108-431b54b6e908 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:46.383232 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1298/2589] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:49.387921 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448e84b-98df-4e0d-8686-0652754bdda8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:49.388460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448e84b-98df-4e0d-8686-0652754bdda8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:49.388460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448e84b-98df-4e0d-8686-0652754bdda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:49.408415 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c448e84b-98df-4e0d-8686-0652754bdda8 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:49.409603 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c448e84b-98df-4e0d-8686-0652754bdda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:49.418264 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1292/2590] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:49.440962 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28870a34-7efc-424a-a108-431b54b6e908 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:49.452639 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28870a34-7efc-424a-a108-431b54b6e908 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:49.460610 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd4edc-f116-4604-bc9a-f36fecd2f7a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:49.461213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd4edc-f116-4604-bc9a-f36fecd2f7a1 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:49.461490 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd4edc-f116-4604-bc9a-f36fecd2f7a1 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:49.461756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd4edc-f116-4604-bc9a-f36fecd2f7a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:49.470802 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd4edc-f116-4604-bc9a-f36fecd2f7a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:49.472388 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd4edc-f116-4604-bc9a-f36fecd2f7a1 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:49.485085 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1299/2591] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:52.562285 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42541f-9726-4709-9efb-80714f4ca359 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:52.562875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42541f-9726-4709-9efb-80714f4ca359 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:52.562875 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42541f-9726-4709-9efb-80714f4ca359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:52.586075 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5e42541f-9726-4709-9efb-80714f4ca359 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:52.587319 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5e42541f-9726-4709-9efb-80714f4ca359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:52.597104 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1293/2592] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:52.624641 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd4edc-f116-4604-bc9a-f36fecd2f7a1 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:52.636776 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd4edc-f116-4604-bc9a-f36fecd2f7a1 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:52.643507 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261bb2b-bb69-40b8-8069-fe7ef5931625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:52.644112 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261bb2b-bb69-40b8-8069-fe7ef5931625 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:52.644357 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261bb2b-bb69-40b8-8069-fe7ef5931625 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:52.644627 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261bb2b-bb69-40b8-8069-fe7ef5931625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:52.653326 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261bb2b-bb69-40b8-8069-fe7ef5931625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:52.654959 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261bb2b-bb69-40b8-8069-fe7ef5931625 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:52.665056 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1300/2593] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:58:56.783442 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98737d25-b85d-4dde-9e09-f1b4fa2d053d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:56.784047 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98737d25-b85d-4dde-9e09-f1b4fa2d053d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:56.784198 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98737d25-b85d-4dde-9e09-f1b4fa2d053d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:56.804824 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-98737d25-b85d-4dde-9e09-f1b4fa2d053d None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:58:56.806027 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-98737d25-b85d-4dde-9e09-f1b4fa2d053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:58:56.815176 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1294/2594] 10.222.0.36 () {58 vars in 1016 bytes} [Mon Oct 24 08:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:58:56.837814 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261bb2b-bb69-40b8-8069-fe7ef5931625 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:58:56.855154 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261bb2b-bb69-40b8-8069-fe7ef5931625 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:58:56.862642 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925a5e7-9380-4d86-818f-6f926663756b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:58:56.863252 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925a5e7-9380-4d86-818f-6f926663756b service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:58:56.863539 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925a5e7-9380-4d86-818f-6f926663756b service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:58:56.863803 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925a5e7-9380-4d86-818f-6f926663756b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:58:56.872409 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925a5e7-9380-4d86-818f-6f926663756b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:58:56.874011 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925a5e7-9380-4d86-818f-6f926663756b service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:58:56.883495 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1301/2595] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:59:06.799034 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=15 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:06.879375 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=30 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:26.051022 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98737d25-b85d-4dde-9e09-f1b4fa2d053d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:26.052666 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-98737d25-b85d-4dde-9e09-f1b4fa2d053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:26.067832 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98737d25-b85d-4dde-9e09-f1b4fa2d053d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:26.073083 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01c45d-dfe3-444d-a5ee-bcbbf920dda5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:26.073690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01c45d-dfe3-444d-a5ee-bcbbf920dda5 service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.073973 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01c45d-dfe3-444d-a5ee-bcbbf920dda5 service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.074248 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01c45d-dfe3-444d-a5ee-bcbbf920dda5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.076069 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ec01c45d-dfe3-444d-a5ee-bcbbf920dda5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:26.088062 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01c45d-dfe3-444d-a5ee-bcbbf920dda5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:26.089867 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01c45d-dfe3-444d-a5ee-bcbbf920dda5 service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:26.101378 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1295/2596] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:59:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:59:26.110972 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925a5e7-9380-4d86-818f-6f926663756b service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:26.121786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925a5e7-9380-4d86-818f-6f926663756b service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:26.135000 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:26.135749 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.136127 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.136455 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] PATH_INFO: `/v3/users/a49947bfa2a5482d919ba0d4744553c7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.137868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49947bfa2a5482d919ba0d4744553c7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:26.140079 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:26.187379 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49947bfa2a5482d919ba0d4744553c7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.194496 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49947bfa2a5482d919ba0d4744553c7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.195212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49947bfa2a5482d919ba0d4744553c7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.204159 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49947bfa2a5482d919ba0d4744553c7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.216056 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-970c34d9-e5b0-4366-a68a-570f344b5962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49947bfa2a5482d919ba0d4744553c7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.221750 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1302/2597] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:59:26 2022] DELETE /identity/v3/users/a49947bfa2a5482d919ba0d4744553c7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:59:26.403816 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01c45d-dfe3-444d-a5ee-bcbbf920dda5 service placement] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:26.414409 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01c45d-dfe3-444d-a5ee-bcbbf920dda5 service placement] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:26.476460 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:26.477193 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.477490 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.477763 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] PATH_INFO: `/v3/projects/0b1b162d022d4c5ebd8c62dcad277727` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.478960 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1b162d022d4c5ebd8c62dcad277727)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:26.480821 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:26.529711 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1b162d022d4c5ebd8c62dcad277727', 'request_id': 'req-a8351333-82ea-46bd-b00a-3fa1db00d244'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.535268 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1b162d022d4c5ebd8c62dcad277727', 'request_id': 'req-a8351333-82ea-46bd-b00a-3fa1db00d244'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.542395 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1b162d022d4c5ebd8c62dcad277727', 'request_id': 'req-a8351333-82ea-46bd-b00a-3fa1db00d244'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.556708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] The token cache is being invalidate because project 0b1b162d022d4c5ebd8c62dcad277727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:59:26.556999 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.557661 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a8351333-82ea-46bd-b00a-3fa1db00d244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:26.559246 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1296/2598] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:59:26 2022] DELETE /identity/v3/projects/0b1b162d022d4c5ebd8c62dcad277727 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:59:26.564212 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d020e65-5a0b-4900-a92b-3fd86c739428 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.564674 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d020e65-5a0b-4900-a92b-3fd86c739428 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.565012 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d020e65-5a0b-4900-a92b-3fd86c739428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.585398 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0d020e65-5a0b-4900-a92b-3fd86c739428 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:59:26.586632 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0d020e65-5a0b-4900-a92b-3fd86c739428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:26.648489 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1303/2599] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:59:26.656316 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4733ae-11ac-4740-a499-521a0ca61c53 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.656595 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4733ae-11ac-4740-a499-521a0ca61c53 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.656892 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4733ae-11ac-4740-a499-521a0ca61c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.677929 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5b4733ae-11ac-4740-a499-521a0ca61c53 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:59:26.679203 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5b4733ae-11ac-4740-a499-521a0ca61c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:26.687247 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1297/2600] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:59:26.691474 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d020e65-5a0b-4900-a92b-3fd86c739428 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:26.701134 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d020e65-5a0b-4900-a92b-3fd86c739428 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:26.705643 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872eaf6-9084-45e9-8680-02d74004baac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:26.706266 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872eaf6-9084-45e9-8680-02d74004baac admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.706543 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872eaf6-9084-45e9-8680-02d74004baac admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.706844 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872eaf6-9084-45e9-8680-02d74004baac admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.707488 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872eaf6-9084-45e9-8680-02d74004baac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 08:59:26.707817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872eaf6-9084-45e9-8680-02d74004baac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:26.709426 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e872eaf6-9084-45e9-8680-02d74004baac admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:26.721020 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1304/2601] 10.222.0.26 () {58 vars in 1153 bytes} [Mon Oct 24 08:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 08:59:26.725700 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4733ae-11ac-4740-a499-521a0ca61c53 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:26.735101 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4733ae-11ac-4740-a499-521a0ca61c53 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:26.740396 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680870e5-2d4c-4335-b273-0e4fa7b53131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:26.740991 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680870e5-2d4c-4335-b273-0e4fa7b53131 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.741272 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680870e5-2d4c-4335-b273-0e4fa7b53131 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.741541 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680870e5-2d4c-4335-b273-0e4fa7b53131 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.742158 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680870e5-2d4c-4335-b273-0e4fa7b53131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:26.744074 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680870e5-2d4c-4335-b273-0e4fa7b53131 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:26.781746 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1298/2602] 10.222.0.26 () {60 vars in 1153 bytes} [Mon Oct 24 08:59:26 2022] POST /identity/v3/projects => generated 397 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 08:59:26.786099 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872eaf6-9084-45e9-8680-02d74004baac admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:26.796350 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e872eaf6-9084-45e9-8680-02d74004baac admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:26.853865 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5fbff-9f5c-4743-ba36-e092d14d5f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:26.854472 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5fbff-9f5c-4743-ba36-e092d14d5f1e admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.854775 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5fbff-9f5c-4743-ba36-e092d14d5f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.855051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5fbff-9f5c-4743-ba36-e092d14d5f1e admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.855689 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5fbff-9f5c-4743-ba36-e092d14d5f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:26.857600 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5fbff-9f5c-4743-ba36-e092d14d5f1e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:26.865681 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:26.881640 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1305/2603] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Oct 24 08:59:26 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 08:59:26.885859 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680870e5-2d4c-4335-b273-0e4fa7b53131 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:26.896391 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680870e5-2d4c-4335-b273-0e4fa7b53131 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:26.902160 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c730678-e115-4d3a-83ef-74742f53d056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:26.902798 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c730678-e115-4d3a-83ef-74742f53d056 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.903078 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c730678-e115-4d3a-83ef-74742f53d056 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.903348 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c730678-e115-4d3a-83ef-74742f53d056 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.903987 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c730678-e115-4d3a-83ef-74742f53d056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:26.905572 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c730678-e115-4d3a-83ef-74742f53d056 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:26.915813 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1299/2604] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:59:26.919705 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5fbff-9f5c-4743-ba36-e092d14d5f1e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:26.930378 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5fbff-9f5c-4743-ba36-e092d14d5f1e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:26.935595 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c4fa8-4d5a-4f53-84e6-25e31e7f1fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:26.936190 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c4fa8-4d5a-4f53-84e6-25e31e7f1fbe admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.936465 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c4fa8-4d5a-4f53-84e6-25e31e7f1fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.936735 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c4fa8-4d5a-4f53-84e6-25e31e7f1fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.937371 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c4fa8-4d5a-4f53-84e6-25e31e7f1fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:26.938984 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c4fa8-4d5a-4f53-84e6-25e31e7f1fbe admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:26.949752 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1306/2605] 10.222.0.26 () {58 vars in 1122 bytes} [Mon Oct 24 08:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 08:59:26.954016 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c730678-e115-4d3a-83ef-74742f53d056 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:26.964520 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c730678-e115-4d3a-83ef-74742f53d056 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:26.970060 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d3469-e305-4988-acf4-f429895d54f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:26.970710 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d3469-e305-4988-acf4-f429895d54f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:26.971016 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d3469-e305-4988-acf4-f429895d54f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:26.971288 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d3469-e305-4988-acf4-f429895d54f3 admin admin] PATH_INFO: `/v3/projects/967f94b6b62b46cd93986f569e214bc8/users/30dc500152be40b1893569df0182b7ad/roles/290368e3589e425e99a9f4dbfe2b3823` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:26.986217 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d3469-e305-4988-acf4-f429895d54f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=967f94b6b62b46cd93986f569e214bc8, user_id=30dc500152be40b1893569df0182b7ad, role_id=290368e3589e425e99a9f4dbfe2b3823)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:26.988864 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d3469-e305-4988-acf4-f429895d54f3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:27.003642 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1300/2606] 10.222.0.26 () {60 vars in 1483 bytes} [Mon Oct 24 08:59:26 2022] PUT /identity/v3/projects/967f94b6b62b46cd93986f569e214bc8/users/30dc500152be40b1893569df0182b7ad/roles/290368e3589e425e99a9f4dbfe2b3823 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:59:27.013756 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b672f-2e83-4dfc-9a2a-f313d5599ec2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:27.014062 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b672f-2e83-4dfc-9a2a-f313d5599ec2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:27.014335 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b672f-2e83-4dfc-9a2a-f313d5599ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:27.042791 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d62b672f-2e83-4dfc-9a2a-f313d5599ec2 None None] MFA Rules not processed for user `30dc500152be40b1893569df0182b7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:59:27.043918 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d62b672f-2e83-4dfc-9a2a-f313d5599ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:27.107594 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1307/2607] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:59:27 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:59:27.121331 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d3469-e305-4988-acf4-f429895d54f3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:27.123230 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ff9d3469-e305-4988-acf4-f429895d54f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:27.129962 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:27.190012 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d3469-e305-4988-acf4-f429895d54f3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:27.194667 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d4b2b-3970-493e-8d6a-b41fef326fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6046d6440af84aa28aab47a018638e18', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:27.195290 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d4b2b-3970-493e-8d6a-b41fef326fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:27.195563 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d4b2b-3970-493e-8d6a-b41fef326fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:27.195829 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d4b2b-3970-493e-8d6a-b41fef326fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:27.205002 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d4b2b-3970-493e-8d6a-b41fef326fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:27.206438 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d4b2b-3970-493e-8d6a-b41fef326fd4 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:27.215749 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1301/2608] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:59:27 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:59:32.275796 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b672f-2e83-4dfc-9a2a-f313d5599ec2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:32.338083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b672f-2e83-4dfc-9a2a-f313d5599ec2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:32.343483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:32.344130 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:32.344401 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:32.344664 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] PATH_INFO: `/v3/users/30dc500152be40b1893569df0182b7ad` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:32.346021 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30dc500152be40b1893569df0182b7ad)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:32.347997 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:32.404434 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30dc500152be40b1893569df0182b7ad'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:32.411526 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30dc500152be40b1893569df0182b7ad'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:32.412184 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30dc500152be40b1893569df0182b7ad'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:32.429721 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30dc500152be40b1893569df0182b7ad'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:32.436316 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:32.447745 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30dc500152be40b1893569df0182b7ad'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:32.457788 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1308/2609] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 08:59:32 2022] DELETE /identity/v3/users/30dc500152be40b1893569df0182b7ad => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:59:32.462385 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e16b6-e2cf-48b8-8850-7bc95700f935 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:32.462891 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e16b6-e2cf-48b8-8850-7bc95700f935 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:32.463015 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e16b6-e2cf-48b8-8850-7bc95700f935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:32.482481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b60e16b6-e2cf-48b8-8850-7bc95700f935 None None] MFA Rules not processed for user `2e8c1f674ea94c1eb2d8a47d1aad00af`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:59:32.483792 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b60e16b6-e2cf-48b8-8850-7bc95700f935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:32.548812 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1302/2610] 10.222.0.26 () {58 vars in 958 bytes} [Mon Oct 24 08:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:59:32.559138 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:32.560782 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:32.619051 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda8a7d-7dcb-424d-97af-32f36ffa0f18 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:32.622903 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca97e5-4582-4344-8b80-303d8aa8aa65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:32.623509 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca97e5-4582-4344-8b80-303d8aa8aa65 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:32.623786 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca97e5-4582-4344-8b80-303d8aa8aa65 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:32.624054 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca97e5-4582-4344-8b80-303d8aa8aa65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:32.633083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca97e5-4582-4344-8b80-303d8aa8aa65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:32.634510 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca97e5-4582-4344-8b80-303d8aa8aa65 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:32.643689 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1309/2611] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:59:32.915516 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e16b6-e2cf-48b8-8850-7bc95700f935 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:32.917153 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b60e16b6-e2cf-48b8-8850-7bc95700f935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:32.929326 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e16b6-e2cf-48b8-8850-7bc95700f935 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:32.934479 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:32.935145 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:32.935421 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:32.935690 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] PATH_INFO: `/v3/projects/967f94b6b62b46cd93986f569e214bc8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:32.936791 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] RBAC: Authorizing `identity:delete_project(project_id=967f94b6b62b46cd93986f569e214bc8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:32.938684 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:32.986730 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967f94b6b62b46cd93986f569e214bc8', 'request_id': 'req-246d909d-da7d-48fd-808c-7974da80219c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:32.992534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967f94b6b62b46cd93986f569e214bc8', 'request_id': 'req-246d909d-da7d-48fd-808c-7974da80219c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:32.998961 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967f94b6b62b46cd93986f569e214bc8', 'request_id': 'req-246d909d-da7d-48fd-808c-7974da80219c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:33.014404 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] The token cache is being invalidate because project 967f94b6b62b46cd93986f569e214bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 08:59:33.014691 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:33.015357 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-246d909d-da7d-48fd-808c-7974da80219c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 08:59:33.016832 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1303/2612] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 08:59:32 2022] DELETE /identity/v3/projects/967f94b6b62b46cd93986f569e214bc8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 08:59:37.822771 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfa12d-6daa-4e84-88b8-679bf2024b5e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:37.823343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfa12d-6daa-4e84-88b8-679bf2024b5e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:37.823343 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfa12d-6daa-4e84-88b8-679bf2024b5e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:37.824923 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1310/2613] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:59:37.831014 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf602ef-b21c-42e9-bdcd-1cb55b64e742 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:37.831474 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf602ef-b21c-42e9-bdcd-1cb55b64e742 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:37.831596 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf602ef-b21c-42e9-bdcd-1cb55b64e742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:37.871704 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cdf602ef-b21c-42e9-bdcd-1cb55b64e742 None None] MFA Rules not processed for user `2254623f523c48f0aa454b85be41df95`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:59:37.872925 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cdf602ef-b21c-42e9-bdcd-1cb55b64e742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:37.885353 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:37.927138 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1304/2614] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:59:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:59:37.939581 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfa12d-6daa-4e84-88b8-679bf2024b5e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:37.941558 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9fdfa12d-6daa-4e84-88b8-679bf2024b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:37.955437 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfa12d-6daa-4e84-88b8-679bf2024b5e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:37.958957 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23fb3a-2c01-4416-9cff-fefcb14b90b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:37.959566 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23fb3a-2c01-4416-9cff-fefcb14b90b5 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:37.959868 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23fb3a-2c01-4416-9cff-fefcb14b90b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:37.960141 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23fb3a-2c01-4416-9cff-fefcb14b90b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:37.967299 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23fb3a-2c01-4416-9cff-fefcb14b90b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:37.968725 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23fb3a-2c01-4416-9cff-fefcb14b90b5 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:37.976340 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1311/2615] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:59:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:59:38.070411 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524541f2-4772-4638-b54e-7303f336ac30 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:38.070745 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524541f2-4772-4638-b54e-7303f336ac30 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:38.071052 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524541f2-4772-4638-b54e-7303f336ac30 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:38.072387 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1305/2616] 10.222.0.26 () {54 vars in 921 bytes} [Mon Oct 24 08:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 08:59:38.077843 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe534d-e7c8-44ae-9d7a-1a8b03407126 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:38.078083 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe534d-e7c8-44ae-9d7a-1a8b03407126 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:38.078385 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe534d-e7c8-44ae-9d7a-1a8b03407126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:38.118179 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2ebe534d-e7c8-44ae-9d7a-1a8b03407126 None None] MFA Rules not processed for user `f257d9b665714bff8b4ceb4fc5dc8387`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 08:59:38.119438 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2ebe534d-e7c8-44ae-9d7a-1a8b03407126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:38.199844 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1312/2617] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Oct 24 08:59:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 08:59:38.211032 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524541f2-4772-4638-b54e-7303f336ac30 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:38.213084 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-524541f2-4772-4638-b54e-7303f336ac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:38.228007 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524541f2-4772-4638-b54e-7303f336ac30 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:38.232270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378776cb-9828-461a-b277-9cdd9335ae48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:38.232880 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378776cb-9828-461a-b277-9cdd9335ae48 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:38.233270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378776cb-9828-461a-b277-9cdd9335ae48 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:38.233558 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378776cb-9828-461a-b277-9cdd9335ae48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:38.245048 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378776cb-9828-461a-b277-9cdd9335ae48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:38.246489 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378776cb-9828-461a-b277-9cdd9335ae48 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:38.258146 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1306/2618] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:59:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:59:40.203300 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe534d-e7c8-44ae-9d7a-1a8b03407126 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 08:59:40.205534 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2ebe534d-e7c8-44ae-9d7a-1a8b03407126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:40.220855 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe534d-e7c8-44ae-9d7a-1a8b03407126 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 08:59:40.227943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513f3eb-3700-4cee-b507-e56645ac624e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 08:59:40.227943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513f3eb-3700-4cee-b507-e56645ac624e service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 08:59:40.227943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513f3eb-3700-4cee-b507-e56645ac624e service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 08:59:40.227943 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513f3eb-3700-4cee-b507-e56645ac624e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 08:59:40.229647 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3513f3eb-3700-4cee-b507-e56645ac624e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 08:59:40.242927 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513f3eb-3700-4cee-b507-e56645ac624e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 08:59:40.244592 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513f3eb-3700-4cee-b507-e56645ac624e service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 08:59:40.259061 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1313/2619] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 08:59:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 08:59:48.253045 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=21 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 08:59:50.252717 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=80,keystone:DELETE=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:00:43.247996 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378776cb-9828-461a-b277-9cdd9335ae48 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 09:00:43.250504 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-378776cb-9828-461a-b277-9cdd9335ae48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:00:43.334898 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378776cb-9828-461a-b277-9cdd9335ae48 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 09:00:43.340481 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c30f9-e885-45f7-9da7-7edd849e2c80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 09:00:43.341089 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c30f9-e885-45f7-9da7-7edd849e2c80 service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 09:00:43.341371 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c30f9-e885-45f7-9da7-7edd849e2c80 service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 09:00:43.341646 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c30f9-e885-45f7-9da7-7edd849e2c80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 09:00:43.343476 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-634c30f9-e885-45f7-9da7-7edd849e2c80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:00:43.355228 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c30f9-e885-45f7-9da7-7edd849e2c80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 09:00:43.356826 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c30f9-e885-45f7-9da7-7edd849e2c80 service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 09:00:43.367074 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1307/2620] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 09:00:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 09:00:53.362820 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=32 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:24.685934 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513f3eb-3700-4cee-b507-e56645ac624e service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 09:01:24.703164 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513f3eb-3700-4cee-b507-e56645ac624e service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 09:01:24.710353 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085abd9-61a3-4f62-8031-11e55bf40669 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f257d9b665714bff8b4ceb4fc5dc8387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 09:01:24.710979 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085abd9-61a3-4f62-8031-11e55bf40669 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 09:01:24.711261 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085abd9-61a3-4f62-8031-11e55bf40669 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 09:01:24.711529 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085abd9-61a3-4f62-8031-11e55bf40669 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 09:01:24.713340 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5085abd9-61a3-4f62-8031-11e55bf40669 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:01:24.774293 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085abd9-61a3-4f62-8031-11e55bf40669 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 09:01:24.775929 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085abd9-61a3-4f62-8031-11e55bf40669 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 09:01:24.785168 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1314/2621] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 09:01:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 09:01:25.034456 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c30f9-e885-45f7-9da7-7edd849e2c80 service placement] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 09:01:25.044270 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c30f9-e885-45f7-9da7-7edd849e2c80 service placement] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 09:01:25.048341 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2254623f523c48f0aa454b85be41df95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 09:01:25.048935 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 09:01:25.049212 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 09:01:25.049482 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 09:01:25.051274 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:01:25.118275 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 09:01:25.119811 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 09:01:25.129807 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1308/2622] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 09:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 09:01:31.196145 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085abd9-61a3-4f62-8031-11e55bf40669 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 09:01:31.209708 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085abd9-61a3-4f62-8031-11e55bf40669 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 09:01:31.216118 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b2074-5e37-43af-84ac-f751a11f2fbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 09:01:31.216794 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b2074-5e37-43af-84ac-f751a11f2fbc service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 09:01:31.217076 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b2074-5e37-43af-84ac-f751a11f2fbc service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 09:01:31.217348 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b2074-5e37-43af-84ac-f751a11f2fbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 09:01:31.219483 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-237b2074-5e37-43af-84ac-f751a11f2fbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:01:31.233772 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b2074-5e37-43af-84ac-f751a11f2fbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 09:01:31.236869 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b2074-5e37-43af-84ac-f751a11f2fbc service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 09:01:31.250964 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1315/2623] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 09:01:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 09:01:31.259581 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 09:01:31.261192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:01:31.284662 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2490a-48cb-402b-82e6-32f38d22e5bb service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 09:01:31.290703 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 09:01:31.291232 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 09:01:31.291534 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 09:01:31.291808 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] PATH_INFO: `/v3/users/a7c4228e92254a65ada3a0a777812103` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 09:01:31.292975 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7c4228e92254a65ada3a0a777812103)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 09:01:31.295060 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 09:01:31.364771 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7c4228e92254a65ada3a0a777812103'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.365447 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7c4228e92254a65ada3a0a777812103'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.372583 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7c4228e92254a65ada3a0a777812103'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.380080 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7c4228e92254a65ada3a0a777812103'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.394817 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7c4228e92254a65ada3a0a777812103'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.400829 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1309/2624] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 09:01:31 2022] DELETE /identity/v3/users/a7c4228e92254a65ada3a0a777812103 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 09:01:31.576210 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b2074-5e37-43af-84ac-f751a11f2fbc service placement] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 09:01:31.587694 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b2074-5e37-43af-84ac-f751a11f2fbc service placement] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 09:01:31.643950 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 09:01:31.644582 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 09:01:31.644853 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 09:01:31.645122 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] PATH_INFO: `/v3/projects/6d2806e58f714af7a7f739a59d3dece1` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 09:01:31.646270 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2806e58f714af7a7f739a59d3dece1)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 09:01:31.648193 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 09:01:31.695688 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2806e58f714af7a7f739a59d3dece1', 'request_id': 'req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.701454 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2806e58f714af7a7f739a59d3dece1', 'request_id': 'req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.707750 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2806e58f714af7a7f739a59d3dece1', 'request_id': 'req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.722246 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] The token cache is being invalidate because project 6d2806e58f714af7a7f739a59d3dece1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 09:01:31.722538 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.723205 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.724662 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1316/2625] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 09:01:31 2022] DELETE /identity/v3/projects/6d2806e58f714af7a7f739a59d3dece1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 09:01:31.728780 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 09:01:31.730192 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:01:31.743407 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f50eb-4088-41e8-bb6b-b77b0bfdbd89 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 09:01:31.803417 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 09:01:31.804044 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 09:01:31.804321 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 09:01:31.804586 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] PATH_INFO: `/v3/users/8030cebfa4324a639e1d813e6aa092ce` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 09:01:31.805740 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8030cebfa4324a639e1d813e6aa092ce)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 09:01:31.807670 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 09:01:31.813163 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:31.857708 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8030cebfa4324a639e1d813e6aa092ce'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.858398 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8030cebfa4324a639e1d813e6aa092ce'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.866310 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8030cebfa4324a639e1d813e6aa092ce'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.877985 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8030cebfa4324a639e1d813e6aa092ce'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.890837 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8030cebfa4324a639e1d813e6aa092ce'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:31.896763 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1310/2626] 10.222.0.26 () {58 vars in 1224 bytes} [Mon Oct 24 09:01:31 2022] DELETE /identity/v3/users/8030cebfa4324a639e1d813e6aa092ce => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 09:01:32.074359 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 09:01:32.075817 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:01:32.093048 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8e3bf-c889-4f25-b531-25f3f68da4ce admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 09:01:32.105634 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:32.151829 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e8c1f674ea94c1eb2d8a47d1aad00af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2f48107e8bb4875908f0d6dc2693154', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 09:01:32.152389 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 09:01:32.152663 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 09:01:32.152930 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] PATH_INFO: `/v3/projects/a49ac070323b4b42a925286738643569` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 09:01:32.154156 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49ac070323b4b42a925286738643569)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 09:01:32.156148 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 09:01:32.205226 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49ac070323b4b42a925286738643569', 'request_id': 'req-19c1eaf9-d0cb-433f-868c-4f1b09536352'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:32.211292 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49ac070323b4b42a925286738643569', 'request_id': 'req-19c1eaf9-d0cb-433f-868c-4f1b09536352'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:32.217619 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49ac070323b4b42a925286738643569', 'request_id': 'req-19c1eaf9-d0cb-433f-868c-4f1b09536352'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:32.232162 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] The token cache is being invalidate because project a49ac070323b4b42a925286738643569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 09:01:32.232470 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:32.233126 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-19c1eaf9-d0cb-433f-868c-4f1b09536352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 09:01:32.234629 dv-cinder-master-3-smb devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1317/2627] 10.222.0.26 () {58 vars in 1233 bytes} [Mon Oct 24 09:01:32 2022] DELETE /identity/v3/projects/a49ac070323b4b42a925286738643569 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 09:01:41.893468 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:01:42.229213 dv-cinder-master-3-smb devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 09:05:49.194649 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 09:05:49.196354 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:05:49.344347 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be68cb-d827-4442-bce1-31e2a1f1d736 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 09:05:49.409432 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b5333-fae1-4602-ad5c-d0c86af7d396 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3749568726644be9ebdf4d87463ac0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0150abead5e3470d9c6cbf9f9c9e4d4a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 09:05:49.410059 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b5333-fae1-4602-ad5c-d0c86af7d396 service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 09:05:49.410359 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b5333-fae1-4602-ad5c-d0c86af7d396 service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 09:05:49.410651 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b5333-fae1-4602-ad5c-d0c86af7d396 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 09:05:49.412654 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c66b5333-fae1-4602-ad5c-d0c86af7d396 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 09:05:49.424753 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b5333-fae1-4602-ad5c-d0c86af7d396 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 09:05:49.426328 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b5333-fae1-4602-ad5c-d0c86af7d396 service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 09:05:49.436306 dv-cinder-master-3-smb devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1311/2628] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Oct 24 09:05:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 09:05:59.432110 dv-cinder-master-3-smb devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}